How does it work? 3D model optimization for e-commerce performance

3D models are a representation of the physical product and as such, there is almost no limit as to the amount of detail you could include. You could map the product to be millimetre precise, to map every bump, to the ever-subtle variations in colour on its surface. It’s frankly incredible how much of the reality of a product we can encode into a digital form.

However, it comes at a cost, for every bump you map, for every millimetre of precision you add you also increase the size of the model and therefore the amount of time it takes to transmit to the shopper who wants to look at it. Nobody wants to be waiting 15 seconds to view the product they have just clicked on.

At ShopAR, one of the most important parts of our job is to use technology to eliminate this trade off. We want the highest fidelity models possible at the lowest possible cost to the person viewing them. They need to load as fast as possible, look just like the real thing and never cause your phone to feel sluggish.

This article dives into some of our principles for doing that effecting and explains how we approach this problem for our clients.

The type of optimisation

Not all forms optimisations are the same, some result in no visual changes at all. These types of optimisations are called ‘lossless’. Others do change the way a product looks this form of optimisation is called ‘lossy’.

Wherever we can do lossless optimisation we will do it, because what this means is that we have found ways to reduce the size of the model without any cost to the experience. This is our ideal, we preserve all the details without the cost.

However, there are also times when we consider lossy optimisation to be worth it. For instance, if we lose detail on the inside of a shoe which you cannot see, then although we have technically lost detail it does not affect the way the product appears to shoppers.

Where do we start from?

We always start with the highest quality possible models. There is also a trade-off to be made here, there is not a lot of point in mapping the acoustic qualities of a pair of shoes but there is a lot of point in getting the colour and the grain on the leather exactly right. We have a detailed specification which explains our requirements for all the categories we support and sets our quality bar. We think this results in the best quality models in the industry.

Once we have the high-quality model, the optimisation work begins:

Mesh Optimisation

You can think of a 3D model being made up of a series of triangles, the more triangles you have the more precise your model could be (note: you can also create more triangles which give you no more information!).

Note in the model above we keep fine details where they are valuable such as the shape of the holes but reduce them when the surface is relatively flat, like the insole.

When we hand build models, the artist will take care of this as part of the creation process. They make trade-offs about which parts of the model need details and which don’t. When we scan items, we use AI with human support for finishing touches to the model.

This gives us the best trade-off between quality and file size keeping the final check to the human eye. This sets us up to perform the next step which is lossless compression.

We make use of industry leading techniques such as Draco compression, which whilst not always 100% lossless it can keep nearly all of the information whilst reducing the file size dramatically.

Once we have this, we perform a final check to make sure we have all the information we need to “paint” the model with the information about the types of materials and their colours.

Texture optimisation

Now that we have the raw geometry of the product, we need to work out how to describe the materials that the product is made of (e.g. leather, metals etc). We use a material library of based on thousands of real-world samples and products where we can describe almost any material used in real life from acetates for glasses through to leathers, rubbers or suedes uses in shoes.

In practice, these are images which are like an unfurled map of the product itself, as if you peeled the outer layer, placed it out flat, and then fitted it into a square.

For instance, in the image below, which is one of these unfurled maps, as you can see it is all the parts of a New Balance 327.

That image contains colour information, which very literally means what colour the items are but there are others which tell us how metallic a part is, how transparent or even how bumpy the surface is.

We do two things with these images; we resize and compress.

Resizing

Every image is made from tiny dots called pixels, if you look at the image which is a really zoomed in version of the part of the image above you can see that there are squares, each one of these squares is a pixel.

We start with images which are around 4,096 pixels tall and 4,096 pixels wide which means there are about 16.7 million pixels to store. That takes up lots of space and so makes the 3D models bigger.

The thing is that very few people have screens that can support that amount of detail and even fewer will view the 3D products at a size big enough to even be able to see it. So, we make the images smaller. We resize them down so that have between 1/2 and 1/4 of the amount. Where there is fine and valuable information, we may do this a little less.

Compression

Once we have the smaller images, we then optimise them even further. If you look again at the image above there are lots of pixels with exactly the same colour. Instead of storing the values multiple times we may only need to store it once. We use a format called WebP which actually does a whole lot more than that but the takeaway is that we find ways to store less duplicate information and so less information overall making the images smaller.

The combination

Combining these techniques creates some truly impressive results. Check out these shoes they are a fully functioning 3D model which looks amazing, delivered to your browser in less than a megabyte. This is sometimes less than the picture they will be replacing, and it has so much more information!

How do we think about optimisation & user experience?

We talked about starting with high quality models, then using a combination of lossless and lossy methods to product 3D models which are of top quality but also extremely efficient. We think this produces an outstanding user experience and we always working to make this even better.

But that is not the only thing we do. We also think about the user experience to make even the shortest of loading times disappear. Our platform will load 3D models and our AR technology in the on-demand so that it all appears seamless to the shopper and does not slow down thier browsing.

Conclusion

Our platform provides the best of both the human eye, technology, experience curation to provide an outstanding 3D and AR shopping experience. One that drives conversion but also one that retailers are proud of.