mirror of
https://github.com/GraphiteEditor/Graphite.git
synced 2026-09-15 14:18:04 +08:00
Update references to the latest tech stack plans
This commit is contained in:
Keavon Chambers
@@ -166,7 +166,7 @@ And then from a development perspective, I am looking forward to accomplishing t
|
||||
|
||||
- Restoring several previous features that were removed during refactors in the past year to a fully working state including Imaginate, snapping, folder bounding boxes, transform pivots, and vector shape boolean operations
|
||||
- Deploying GPU-based rendering by default and moving from an experimental to a production-ready hardware-accelerated compositing system using [Vello](https://github.com/linebender/vello) to unify the currently separate raster and vector pipelines
|
||||
- Shipping desktop apps for Windows, Mac, and Linux by integrating [Tauri](https://tauri.app/) and bundling built-in AI models to run Imaginate and other upcoming features directly on user hardware
|
||||
- Shipping desktop apps for Windows, Mac, and Linux and bundling built-in AI models to run Imaginate and other upcoming features directly on user hardware
|
||||
- Designing a new vector graphics data format suitable for advanced procedural editing and rendering, plus the associated procedural workflow features
|
||||
- Remaking the Brush tool with the GPU-accelerated pipeline and the adaptive resolution system so digital painting in Graphite becomes practical
|
||||
- Implementing the Mask Mode feature for Magic Wand tool marquee selections, which will dramatically improve Graphite's utility as a raster graphics editor
|
||||
|
||||
@@ -173,7 +173,7 @@ There are so many plans that I'm eager to carry out to improve the clarity and c
|
||||
|
||||
Starting out with our most in-demand request: a desktop app. This has been on our roadmap from the start but only recently it's begun making sense putting it at the front of the roadmap priorities. We hoped to complete it by the end of 2024, but that wasn't in the cards due to developer availability and the specialized skills needed to complete the task.
|
||||
|
||||
Now to get technical, the lazy option exists: chucking the whole web app in an unaltered Electron wrapper, but this is a technological dead end that I believe offers no value compared to a PWA. The value comes from offering an actual native app where the editor Rust code and GPU-accelerated rendering runs on a user's Windows, Mac, or Linux machine without browser overhead. <a href="https://tauri.app/" target="_blank">Tauri</a> provides this capability, but our use case presents some unsupported challenges that our team has to overcome— individually on each platform. The natively rendered viewport content must be composited with the rest of the Tauri window's editor GUI. When investigating this earlier in the year, this isn't properly supported and we will have to develop solutions to fix Tauri's use of the OS platform APIs. If you have experience with native development on Windows, Mac, and/or Linux, please get involved to speed up this effort! With our current resources, I am anticipating this will be ready for release around spring.
|
||||
Now to get technical, the lazy option exists: chucking the whole web app in an unaltered Electron wrapper, but this is a technological dead end that I believe offers no value compared to a PWA. The value comes from offering an actual native app where the editor Rust code and GPU-accelerated rendering runs on a user's Windows, Mac, or Linux machine without browser overhead. Our use case of combining the web-rendered editor interface with the user's native-rendered artwork presents several unique challenges that our team has to overcome— individually on each platform. If you have experience with native development on Windows, Mac, and/or Linux, please get involved to speed up this effort! With our current resources, I am anticipating this will be ready for release around spring.
|
||||
|
||||
### Animation
|
||||
|
||||
|
||||
@@ -41,7 +41,7 @@ Out of the [full list of project opportunities](/volunteer/guide/student-project
|
||||
- Creating an efficient brush rendering system for large painted path datasets
|
||||
- Managing colors through flexible use of color spaces, color models, and color profiles
|
||||
- Researching an assortment of algorithms for image processing operations
|
||||
- Engineering solutions for combining Tauri's webview and Graphite's WGPU renderer in the same window across Windows, Mac, and Linux desktop applications
|
||||
- Engineering solutions for combining our web-based editor GUI and native WGPU renderer in the same window across Windows, Mac, and Linux desktop applications
|
||||
|
||||
These recently added projects are pending an extended description, but we'll be happy to discuss the details if you pop into our [Discord server](https://discord.graphite.rs) and introduce yourself with links to related experience and projects.
|
||||
|
||||
|
||||
@@ -15,17 +15,6 @@ The best introduction for getting up-to-speed with Graphite contribution comes f
|
||||
<img data-youtube-embed="vUzIeg8frh4" src="https://static.graphite.rs/content/volunteer/guide/workshop-intro-to-coding-for-graphite-youtube.avif" onerror="this.onerror = null; this.src = this.src.replace('.avif', '.png')" alt="Workshop: Intro to Coding for Graphite" />
|
||||
</div>
|
||||
|
||||
<!-- ## Tech stack -->
|
||||
<!-- - rustc: Compiler for node graph generics and custom nodes -->
|
||||
<!-- - rust-gpu: Compiler backend to generate compute shaders from Rust source code -->
|
||||
<!-- - wgpu: Portable graphics API for running compute shaders on desktop and web -->
|
||||
<!-- - Tauri: lightweight desktop web UI shell while the backend runs natively (experimental) -->
|
||||
<!-- - Vello: GPU-accelerated vector graphics renderer -->
|
||||
<!-- - COSMIC Text: Text shaping and typesetting -->
|
||||
<!-- - Wasmer or Wasmtime: Portable, sandboxed runtime for custom nodes -->
|
||||
<!-- - Tokio: parallelized job execution in the node graph pipeline -->
|
||||
<!-- - Xilem: High-performance native UI framework, to replace Tauri when ready -->
|
||||
|
||||
## Codebase structure
|
||||
|
||||
Graphite is built from several main software components. New developers may choose to specialize in one or more area without having to attain a working knowledge of the full codebase.
|
||||
|
||||
@@ -62,7 +62,7 @@ The fully compiled regime is used only when the user exports the procedural artw
|
||||
|
||||
### Compile server
|
||||
|
||||
The three regimes have thus far been only a description of the eventual architecture direction. The interpreted regime is currently the only mode implemented in Graphene. The other two will require access to `rustc` which will necessitate the compile server that we will finish building and then publicly host for Graphite users in the future. Users of the desktop version of Graphite, utilizing [Tauri](https://tauri.app/), will be able to use an embedded `rustc` if the user has opted to download the Rust toolchain while installing Graphite.
|
||||
The three regimes have thus far been only a description of the eventual architecture direction. The interpreted regime is currently the only mode implemented in Graphene. The other two will require access to `rustc` which will necessitate the compile server that we will finish building and then publicly host for Graphite users in the future. Users of the desktop version of Graphite will be able to use an embedded `rustc` if the user has opted to download the Rust toolchain while installing Graphite.
|
||||
|
||||
Without a compile server, all the nodes are precompiled when Graphite is built. The node registry (in the file `node_registry.rs`) currently exists to allow the interpreted executor to find the Rust functions that correspond to each node with its appropriate type signature. Nodes support generics, so it's currently necessary to list every forseeable concrete type signature in the registry until the compile server can generate bytecode for less common type combinations on-the-fly.
|
||||
|
||||
|
||||
@@ -26,8 +26,6 @@ cargo install -f wasm-bindgen-cli@0.2.100
|
||||
|
||||
Regarding the last one: you'll likely get faster build times if you manually install that specific version of `wasm-bindgen-cli`. It is supposed to be installed automatically but a version mismatch causes it to reinstall every single recompilation. It may need to be manually updated periodically to match the version of the `wasm-bindgen` dependency in [`Cargo.toml`](https://github.com/GraphiteEditor/Graphite/blob/master/Cargo.toml).
|
||||
|
||||
Lastly, if you intend to develop using the Tauri desktop app build target, obtain [Tauri's dependencies](https://v2.tauri.app/start/prerequisites/). This is not the usual setup for most contributors, so you will know if you need it.
|
||||
|
||||
## Repository
|
||||
|
||||
Clone the project to a convenient location:
|
||||
|
||||
@@ -83,7 +83,7 @@ AI/ML is filling a rapidly growing role as a tool in the creative process. Graph
|
||||
<details>
|
||||
<summary>For additional technical details: click here</summary>
|
||||
|
||||
The approach should be extensible to future models. It needs to run fast and natively on the assorted hardware of local user machines with hardware acceleration. It should be a one-click installation process for users to download and run models without requiring dependencies or environment setup. Ideally, it should allow the more lightweight models to run locally in browsers with WebGPU. It needs to also be deployable to servers in a scalable, cost-viable manner that reuses most of the same code that runs locally. Runtime overhead, cold start times, and memory usage should be minimized for quick, frequent switching between models in a node graph pipeline. The tech stack also needs to be permissively licensed and, as much as possible, Rust-centric so it doesn't add complexity to our Wasm and Tauri build processes. For Stable Diffusion, we need the flexability to track the latest research and extensions to the ecosystem like new base models, checkpoint training, DreamBooth, LoRA, ControlNet, IP-Adapter, etc. and expose these functionalities through modular nodes.
|
||||
The approach should be extensible to future models. It needs to run fast and natively on the assorted hardware of local user machines with hardware acceleration. It should be a one-click installation process for users to download and run models without requiring dependencies or environment setup. Ideally, it should allow the more lightweight models to run locally in browsers with WebGPU. It needs to also be deployable to servers in a scalable, cost-viable manner that reuses most of the same code that runs locally. Runtime overhead, cold start times, and memory usage should be minimized for quick, frequent switching between models in a node graph pipeline. The tech stack also needs to be permissively licensed and, as much as possible, Rust-centric so it doesn't add complexity to our Wasm and desktop build processes. For Stable Diffusion, we need the flexability to track the latest research and extensions to the ecosystem like new base models, checkpoint training, DreamBooth, LoRA, ControlNet, IP-Adapter, etc. and expose these functionalities through modular nodes.
|
||||
|
||||
To meet most of these criteria, our current thinking is to distribute and run our models using the [ONNX](https://onnx.ai/) format. This would integrate ONNX runtimes for WebGPU, native (DirectML, CUDA), and GPU cloud providers. The issue with this approach is that these models (particularly Stable Diffusion) aren't available in an ONNX format.
|
||||
|
||||
|
||||
Reference in New Issue
Block a user