TouchDesigner Point Clouds and 3D Gaussian Splatting

A cloud is a very funny art form. It is like a re-opposing particle sculpture after the real world was scanned: far as it looks like fog, near as a moist, each occupying a few spots in space. It is not a model of integrity, smoothness, closure in the traditional sense, and therefore always carries a sense of instability; but it is also because of this instability that it is particularly suited to visual expression, especially to the illusions that want to retain both “real sense” and “digital sense”.

The questions in this article can also be addressedFrom ComfyUI to LibTV: What capacity should workstream programming grow in the age of video generation?、How the game industry is introduced AI AgentHow the concept of a relatively close read together is developed in different contexts.

If Blender or Houdini are more like a three-dimensional object in fine-tunnel construction, then TouchDesigner makes me feel more like organizing a real-time visual system. It is concerned not only with what a model will look like in the end, but also with how it will be driven, distorted, and work with sound, sensors, time, weather and even in the field. A cloud fits the line: it's not the surface of death, it's a bunch of data that can be manipulated in real time.

Touch Designer, what is it?

From the definition of the official product page of Derivative, TouchDesigner is a visual development platform: It is a real-time, nodal, visual development tool suitable for interactive media systems, architectural projection, live music visualization and fast prototypes. It places the signal flow, the image stream and the geometric stream of the visual system in front of you, so that you can look at the results while you're in the system.

That's why it's particularly good for clouds. It's a Derivative. Point Clouds Directly mentioned in the document, the cloud in the TouchDesigner is often most effectively handled by placement in the GPU texture: a pixel can correspond to the location or properties of a point.Point File In TOP and Point File Select TOP It also directly imports common cloud files, tearing colours, positions and other properties into data layers that can continue to be processed. In TouchDesigner, the cloud is not a static model, but a cluster of data that can continue to be deformed, mixed, map and driven.

What can I do in Touch Designer?

This section does not begin with no nodes or process (and I am not familiar with it). The most attractive place for a cloud in Touch Designer is where it can turn models into something more like fog, constellations, building debris and laser sculptures. It is not necessarily complete, nor is it stable; it can blink, disperse, recombine, drift, even like a luminous mass driven by sound and space.

1. Large space: cities become light particles

Official case of Derivative Point Cloud Mastery from Think and Sense It's a good place to put it at first sight: real neighborhoods become a luminous space dust, like city scans and a huge digital map.

The city on the screen is a cloud of lighted space.

Far from being a building, near being a floating spot, the picture is naturally carrying a sense of “realism is being dismantled”.

Dissolution and reorganization: the beginning of the melting of the boundary

One of the best shows of the cloud is to put the picture between "to be shaped" and "to be disintegrated." The boundaries of traditional models are usually hard, while the boundaries of the clouds can naturally become density, noise and light spots.

When the construction boundary is dispersed, space is more like a layer of light air.

The level, brightness and proximity of the cloud will push the picture to a stronger stage.

Transition and outbreak: images breathing

The advantage of TouchDesigner is to move from static rendering to a live vision that will change continuously. The dots can be compact, dense, from contours to clouds, from space debris to abstract rays.

When the clouds are passing, the images are sliding from one space to another.

The same set of spatial materials can be pushed into a stronger sense of light and motion.

4. Abstract sculpture: point as visual energy

Of course, a cloud does not always come from a reality scan. It can be directly abstracted: like a nebula, like a sandstorm, like a digital sculpture that is being shaped by noise. It's not “what it represents”, it's the density, speed and light that it creates on the screen.

Generating dot clouds is more like a cluster of digital nebulas that are condensing.

It can turn the real world into a more flairful and unstable visual state: one second like a city, one second like a fog, one second like a group of stars that are bursting.

Where did these clouds come from?

If such projects are really going to be carried out, there are probably three main sources of material: self-modelling, photogrammetry, 3D Gaussian Splatting.

Route	Threshold	Realism.	Subsequent redactionability	What's the best for you?
Model yourself.	Top, need to have a clear modelling capability	Controllable, but the real world is a manual detail.	The strongest, the clearest structure.	Projects that require strong control, a sense of design and follow-up to the production of animation and asset management
Photogrammetry	Medium, need to film the norms and reconstruction process	It's strong, especially for architecture, scenery, sculpture.	Medium, often clean, surface, change format	I want to bring real space into the system quickly, but still want to keep the traditional 3D pipe interfaces.
3DGS	One of the lowest, the collection threshold has dropped significantly.	Very strong, especially for the preservation of light, reflection and space. Round	Weakness, not for fine editing.	Atmospheric capture, spatial memory, rapid sampling of the real world, installation of visual materials

The best way to model yourself is, of course. You know where every face, every level, every UV is, how it's convenient to change at a later stage. But its problems are also obvious: slow, and the occasional details of the real world are not very friendly. A lot of TouchDesigner's cloud-drived places are not the standard of geometry, but the tumultuous, but real space information in reality.

Photogrammetry is a more convenient route. The case of Think and Sense is typical: first, a drone image, then a photogrammetry software to restore a cloud and then to TouchDesigner. The advantage is that you still work in a more familiar three-dimensional re-engineering logic, and that what is exported last is easier to access to the existing graphic processes, but still at a cost.

And 3D Gaussian Spratting is more like another door that has been opened suddenly in recent years.The real world no longer has to be gridd before it can enter the real-time visual system. And 3D Gaussian Spratting is born close to the clouds.

Inria Official 3D Gaussi Splatting Project page example: This expression is more suited to preserving volume, light and spatial continuity in the real scene.

And more realistically, the 3DGS are not just about reading in research papers. The tool, represented by Scanivers, has put Gaussian Splatting in the mobile collection stream. Learn a full set of heavy 3D software before trying to move the real world into it, which is no longer the only entry point. You can capture a space and decide whether it ends up as a file, a browse, or a TouchDesigner, into a visual material on the ground.

Why do I think 3DGS has a special potential?

And the most interesting thing about 3DGS and TouchDesigner is not that it's more real, but that the real-world space suddenly becomes a material that can be manipulated in real time.

You can sweep a room, a street, a gallery, and get a big-ass spot with light, reflection, air sense and depth. It does not need to be transformed into a clean model, nor needs to be completely organized as traditional assets. It can enter the TouchDesigner as a piece of the real world, then be stretched, dispersed, rotated, scintillated, vibrated with music, or re-assembled into a space as the audience approaches.

It's funny: 3 DGS gives you a sense of reality (although not so precise), TouchDesigner gives you real-time performance skills. The former is responsible for taking the world down, while the latter is responsible for re-emerging, breaking and deforming the world. The last thing that comes out is not just a 3D scan, but a light material between memory, space and stage vision. The change in the cloud has instead captured the 3DGS shortboard: inaccurate, but fun enough.

3DGS transforms daily scenes into space materials that are visible, can be travelled and can be continued.

3DGS may not be able to be more like a real model, but...The government is not only collecting reality, breaking reality, but also performing reality in real time. That's where it's interesting to put with Touch Designer.

Official case and video

References

Derivative, Created with TouchDesigner
Derivative Docs, TouchDesigner Main Page
Derivative Docs, Point Clouds
Derivative Docs, POP
Derivative, Point Cloud Mastery from Think and Sense
Derivative, Generative Point Clouds in TouchDesigner
Derivative, Audio Reactive 3D Point Clouds in TouchDesigner
Inria, 3D Gaussian Splatting for Real-Time Radiance Field Rendering
Scaniverse, Home
Scaniverse, How to use Scaniverse 3D Scanner for iOS and Android