Unity DLC system implementation tutorial. Problems and features overview. | СЕРГЕЙ ТАРАБАН Блог разработчика игр

In this article I’ll tell you how I implemented Downloadable content system (DLC system) in Unity for one of my game and overview some problems and features.

Author: Sergey Taraban

(Russian version of this article)

Statement of the problem

There is a shop in game. Player buy items there using in-game or real currency. Shop has about 200 items. When player enters game the first time it has only 20 items. If network connection available game downloads DLC in the background. Also, we have a level data in our DLC. Every game level have textures and .asset files. New location also should be added via DLC. Resource loading from DLC bundle should be synchronous. Platform: iOS (iPhone 3GS and hight) and Android (Samsung Galaxy S and hight).

What content DLC should contain and how to manage it.

The game items are completely determined in the file itemdata.txt, which contains information about things and their textures. So, in each DLC bundle should be itemdata.txt file with a set of those things that are in the DLC + texture units for these things. And, when the store will ask the database things Merge all text files with all the DLC and give him the file.

Similarly, the locations locationdata.txt file contain list of characteristics and locations + textures and asset files for them.

The corresponding C # code for loading resources in the game logic:

public String GetItemDataBase() {
    if(DLCManager.isDLCLoaded() == true) {
        //stick all the files in all downloaded DLC and return as a string
        String itemListStr = DLCManager.GetTextFileFromAllDLCs(“itemdata”);
        return itemListStr;
    }
    else {
        //load file by default
       TextAsset itemTextFile = Resources.Load(“itemdata”) as TextAsset;
       return itemTextFile.text;
    }
    return String.Empty;
}

Similarly for texture loading request we check his availability in DLC bundle. If it there we load it and if not load it from local game resources or some default texture in worse case.

public Texture GetTexture(string txname) {
    Texture tx = null;
    if(DLCManager.isDLCLoaded() == true) {
        tx = DLCManager.GetTextureFromDLC(txname);
    }
    if(tx == null) {
        tx = Resources.Load(txname) as Texture;
    }
    if(tx == null) {
        Assert(tx, “Texture not find: ” + txname);
        tx = Resources.Load(kDefaultItemTexturePath) as Texture;
    }
    return tx;
}

Similarly for .asset files we have GetAsset(string assetName) function. It has the same implementation so we skip it.

DLC bundle file.

We have knew now about content of our DLC bundle. It remains to define the form of content storing for DLC bundle.

First variant – store DLC as zip archive. In every archive – text file + N textures. Textures should be in PVRTC format for saving video memory (VRAM). Here we got out first problem – Unity supports textures loading from the file system only in PNG or JPG format [link]. Then you may write pixel data to PVRTC texture [link]. This is slow process because you need encode pixel data to PVR format in real-time. Inside DLC we are planning to store .asset files, and may be scene files (.scene) in the future. So this method not for us.

Second variant – using AssetBundle feature. According to documentation AssetBundle have a lot of useful features:

May contain any Unity data including PVRTC compressed texture (right up our street).
Archive with good compression
Simple and easy to use
Support version control and hash sum (when loaded by LoadFromCacheOrDownload)

The one minus is AssetBundle required Unity Pro version and did not support encryption. We decided to choose this solution because it obviously more attractive and let us solve our issues.

Implementation.

For a start I create basic working version of DLC system. All 200 item’s textures and level files was packed into one AssetBundle and transferred to remote server. File have 200 mb size. I packed AssetBundle by my Unity Editor script for asset bundle generation.

After start application we made this steps:

First we need to download DLC from remote server. According to Unity manual write downloaded files into file on disk for future usage:

// Start a download of the given URL using assetBundle version and CRC-32 Checksum
WWW www = WWW.LoadFromCacheOrDownload (urlToAssetBundle, version, crc32Checksum); 
// Wait for download to complete
yield return www;
// Get the byte data
byte[] byteData = www.bytes;
// Here you may add your custom description method
byteData = MyDescriptionMethod(byteData);
//save byteData in file with .unity3d extension
…
// Frees the memory from the web stream
www.Dispose();
//DLC have been loaded successfully and now we can use it in game
DLCManager.SetDLCLoaded(true);

In this code we are likely to get a c low memory crash in iPhone 3GS, because class WWW not support buffered downloading and stores all loaded information in memory. We’ll talk about this issue a little later. While remember this moment and move on.

Loading DLC data. Now we need to inplement GetTextureFromDLC(), GetAssetFromDLC() и GetTextFileFromAllDLCs(). Definition of last ones I ommit because of it has no different from the first function except of resource it working with.

Main function of GetTextureFromDLC is synchronous loading texture by name from DLC. I defined it here:

public Texture GetTextureFromDLC(String textureName) {

//load DLC file from file system. We can use only synchronous method
AssetBundle asset = AssetBundle.CreateFromFile(pathToAssetBundle);

//synchronous texture loading from DLC bundle
Texture texture = asset.Load(textureName) as Texture;

//unloading DLC bundle from memory without deleting texture object
asset.Unload(false);

return texture;
}

The code above is the only one possible way to load a resource synchronously from AssetBundle. And there is lots of nuances. Let me overview them in order.

AssetBundle.CreateFromFile function according to documentation loads asset from file system synchronously. But “Only uncompressed asset bundles are supported by this function”. Thus, it is only possible to load uncompressed AssetBundle synchronously. This will significantly increase traffic and loading time of DLC from the server. In addition, Unity does not support conversion of AssetBundle compressed to uncompressed, so it is impossible get to download a compressed bundle, and then decompress it on the client side.

The reader may wonder , why not download AssetBundle asynchronously , for example, by function LoadFromCacheOrDownload, and then just take required resources simultaneously from it. After all, it is logical that AssetBundle, loaded from the file system, only loads the file header, and therefore should not takes a lot of memory.

However, this was not the case. Loaded AssetBundle stored in memory with all content in uncompressed form. Thus, to load a one texture from 200, Unity loads all textures into memory, give you one and then release the memory for other 199 textures. We found this out experimentally by measurements of the memory on the devices. It is unacceptable for mobile devices.

Summary

Implementation below is the only one way we found to implement a synchronous download DLC and resources from it. It requires uncompressed AsssetBundle and, as a result, large losses of time and bandwidth when downloading DLC. Option is suitable for relatively small AssetBundle-s, as consumes a lot of memory.

Work on the mistakes.

Let's try to take into account all the previous problems and find solutions for them.

We may solve problem with loading of huge AssetBundles by two ways: Проблема с загрузкой больших assetBundle-ов можно решить двумя способами. First way – by using WebClient class. But I have a problem with it in iOS. WebClient not able to download anything in iOS, but on desktop it work properly. Second way – use native iOS function. For example, NSURLConnection for iOS or URLConnection for Android, which support buffered downloading to file on disk.

But it is not big problem because we need to decrease loading time.

More serious problem is synchronous AssetBundle download. Is hould be uncompressed and take little place in memory. So we need to divide our huge asset bundle to many separate parts. However, if we divide by too small files, there will be many ones, and it would greatly increase the load time. So we still have to keep them compressed for a better time-saving and traffic consumption.

To solve this problem it was decided to use external compressed library. Was chosen an open library compressor 7zip for C#.

The next our steps:

Create AssetBundle with option BuildOptions.UncompressedAssetBundle.
Then bundle was encripted end compressed by activator and was uploaded to the remote server.
During game working I created separate game thread in background. It download DLC bundles from server, uncompress them and same in folders in file system.

Here we got one more problem. Because we use compressed AssetBundle we cannot use LoadFromCacheOrDownload function. So, now we have to define our own version control system for DLC.

On server in folder with DLC we made file named dlcversion. It contains all DLC files names in this folder and md5 hash of them. We compute this hash when upload file on server. On client side we have the same file and on start client compare this file with file on server. If some node in client dlcversion was different from server, then we upload new DLC from server.

Once the new DLC file was downloaded and unzipped his hash once again checked with the server hash, and only after that we rewrite client DLC file by new downloaded DLC from server.

This system has been successfully implement and it works well. The only negative we had is a small drawdown of fps (lags) when downloading and unpacking DLC in the backgrounds. And also slightly increased peak memory consumption of the application.

Thanks for reading. I will be glad to answer your questions.