Art Pipeline Part 3

Posted On: 2019-02-04

By Mark

This is part 3 in a series about the art pipeline I am building. If you haven't already read it, I recommend checking out part 1and part 2. Part 1 is dedicated to describing the "what" and the "why", while Part 2 describes in depth how I achieved some of those goals by automating Gimp. This section will be focused on achieving the remaining goals in Unity: default import settings and automated material generation. Similar to Part 2, this section will include all the relevant code along with detailed descriptions.

Import Settings

When you first add art into a Unity project, Unity will automatically import the file according to its defaults, which have historically been based on which template you used to create the project (3D projects import images as textures, 2D projects import images as sprites). Since my project is 2D but uses textures on quads, each file imported needed to be changed from Sprite to Texture2D. Additionally, since I am working with images at their intended resolution, I needed to configure additional settings, such as point filter mode and disabling mip-maps.

Fortunately, newer versions of Unity (2018.1 and newer) have a feature called Presets which provide an easy way to store an asset's import settings, and then re-use them in future imports. To use this feature, all you need to do is configure one asset the way you want it, and then click the slider(?) icon in the top right (the strange icon between the manual and the gear icons.) That will allow you to "Save Current To..." which will store the preset in a file. (The documentation clearly describes how to do this, complete with pictures, so check that out if any of this is unclear.) Additionally, when you look at an import preset in the inspector, a helpful "Set as TextureImporter Default" button appears: this will replace the built-in defaults with the preset that you've chosen. In my case, I used that to define all art imports to be 2D textures.

_{Screenshot of my current default settings. I disabled mip-maps and use point filter for the art since the camera will always be a fixed distance from the content.}

Normal maps

Normal maps require different settings from regular textures, but are also less common than regular textures (so using them as the default doesn't make sense). Fortunately, it is possible to use a preset inside an AssetPostProcessor to set the asset preset as desired. The Unity documentation provides an example that uses folder names, and it was pretty straightforward to adjust this to using file names instead. Before getting into the example, it is worth mentioning that this is an editor-only script, so it must be stored in a folder named "Editor". In my case, I created one Editor folder as well as one sub-folder (which is where I store all my art-related editor scripts).

To make an asset post processor, create a class that inherits from the AssetPostProcessor class:

public class ApplyImportDefaults : AssetPostprocessor {

After that, add a method OnPreprocessTexture, which Unity will automatically call any time a texture asset is added to the project:

public void OnPreprocessTexture() {

Since this will run on every import, it's important to minimize the amount of work it performs on unrelated textures. To accomplish this, I defined a convention for myself: art that should be imported this way will be in a folder (or sub-folder) called "Character". Additionally, since I may need to change the name later, I used a constant to make it easy to swap the name (for example, if I add normal maps to backgrounds, it wouldn't make sense to put them in a folder called "Character", so I would need to use a different name):

private const string WatchedFolderName = "/Character/";
 
public void OnPreprocessTexture()
{
    if (assetImporter.assetPath.Contains(WatchedFolderName)) {

Once we've determined that we should be modifying this texture, we can get into the meat of the code: setting which preset to use. Since I use Sprite Illuminator I have the convention that "_n" is appended to the filename when I export the normal maps, so I hardcoded that convention:

var importer = (TextureImporter)assetImporter; 
if (name.EndsWith("_n.png"))
{
   //Is a normal map

Finally, the logic for applying a preset is fairly simple: load the preset and call ApplyTo on the importer. Since I may need to do this for a variety of different kinds of presets, I made it a seperate method:

private void ApplyPreset(TextureImporter importer, string presetPath)
{
    var preset = AssetDatabase.LoadAssetAtPath<Preset>(presetPath);
    if (preset == null)
        Debug.LogError("Unable to find required preset at path " + presetPath);
    else
        preset.ApplyTo(importer);
}

This ApplyPreset is called inside the OnPreprocessTexture after determining whether or not it is a normal map:

//Is a normal map 
ApplyPreset(importer, NormalMapTexturePresetPath);

That is really all there is to the code. There are, of course, little details like the right import statements (presets have their own namespace), but none of those are really worth mentioning. I'll just include the complete code here, in case you need those details (just click Expand Code):

//The #if UNITY_EDITOR preprocessor makes sure that this code only ever runs inside the editor 
//  In theory the folder name "Editor" should ensure that, but this is a safeguard against the file getting moved or copied elsewhere.
#if UNITY_EDITOR
using System.IO;
using UnityEditor;
using UnityEditor.Presets;
using UnityEngine;
 
public class ApplyImportDefaults : AssetPostprocessor {
    //Hardcoded for now, but ideally would like to have these more dynamic
    private const string StandardTexturePresetPath = "Assets/ImportDefaults/TextureImporter_Default.preset";
    private const string NormalMapTexturePresetPath = "Assets/ImportDefaults/TextureImporter_NormalMap.preset";
 
    //The texture import code will run on any asset that is inside a folder with this name (either directly or in any number of subfolders)
    private const string WatchedFolderName = "/Character/";
 
    public void OnPreprocessTexture()
    {
        if (assetImporter.assetPath.Contains(WatchedFolderName))
        {
            var name = Path.GetFileName(assetPath);
 
            //Report to the user that we are running the import
            Debug.Log("Importing asset: " + name);
 
            var importer = (TextureImporter)assetImporter;
            if (name.EndsWith("_n.png"))
            {
                //Is a normal map
                ApplyPreset(importer, NormalMapTexturePresetPath);
            }
            else
            {
                //Use regular texture settings
                ApplyPreset(importer, StandardTexturePresetPath);
            }
        }
    }
 
    private void ApplyPreset(TextureImporter importer, string presetPath)
    {
        var preset = AssetDatabase.LoadAssetAtPath<Preset>(presetPath);
        if (preset == null)
            Debug.LogError("Unable to find required preset at path " + presetPath);
        else
            preset.ApplyTo(importer);
    }
}
#endif

Making Materials

Once all the textures are imported, I need to create materials for each one before they can be used in the game. To automate this, I decided to write a simple script that I can run from the menu, to create materials for any textures in the selected folder (or sub-folders). Additionally, since some of the required textures may be missing (for example, omitting normal maps until a later polish phase) the script is also capable of modifying existing materials to add in any newly added textures as well.

Creating a script that can be run from the menu is relatively simple: it requires a static method decorated with the MenuItem attribute. For simplicity, I created a new static class in the editor folder (right next to the ApplyImportDefaults.cs) that will contain the functionality. For the path to the menu item, I use a (new) sub-folder in the "Tools" menu called "Art Pipeline". This allows me to easily group together any art automation.

public static class MaterialGeneration
{
  [MenuItem("Tools/Art Pipeline/Create Materials In Folder")]
  public static void MakeMaterialsFromFolder()

Before diving into the code, I have a couple brief asides to mention here. Firstly, the code in this script is a conglomeration of a lot of patterns and approaches that I found online. Unfortunately, I only kept notes on the source of ready-made things (which were few) so I can't properly cite all the various sources of code that I had to bend to meet my needs. Secondly, there were a few activities that I isolated into their own static methods, in a new class (called EditorUtilities, not to be confused with Unity's EditorUtility class - I should probably pick a different name...) Those separate methods are generic enough that they should be able to be used in other forms of automation, so they are in a seperate class to account for that. The two methods GetValidatedSelectionand GetLeafDirectoriesare each responsible for a single task.

GetValidatedSelectionis a simple helper method that performs simple validation on the user's current selection, to make sure that the user selected something in the project (as opposed to an object in the scene, or having no selection at all.) It's flexible enough to handle multiple selected items, and will return any that pass validation (while logging warnings for any that do not.)

/// <summary> 
/// Yields a list of assets the user has selected in the editor. Items that are not assets will be omitted (and the user will see a warning for each item omitted).
/// </summary>
/// <param name="activityDescription">A description of the activity that requires the selection (used in error messages.)</param>
public static IEnumerable<Object> GetValidatedSelection(string activityDescription)
{
    if (Selection.objects.Length == 0)
    {
        Debug.LogWarning("Selection required before " + activityDescription);
        yield break;
    }
    foreach (Object o in Selection.objects)
    {
        if (!AssetDatabase.Contains(o))
        {
            Debug.LogWarning("Asset Database does not contain " + o.name);
            continue;
        }
        yield return o;
    }
}

GetLeafDirectories is a helper method that searches sub-folders of a current item to find all folders that do not contain any sub-folders. It uses Unity's Asset Database to navigate the folders, though the same pattern (just different method calls) would work for searching the file system.

/// <summary>
/// Recursively walks all directories in the specified path. Returns the asset path for any folders that do not contain any sub-folders (a "leaf" on the tree.)
/// </summary>
public static IEnumerable<string> GetLeafDirectories(string path)
{
    var subfolders = AssetDatabase.GetSubFolders(path);
    if (subfolders.Length > 0)
    {
        foreach (var subfolder in subfolders)
        {
            foreach (var result in GetLeafDirectories(subfolder))
            {
                yield return result;
            }
        }
    }
    else
    {
        //This is a leaf node, return it here.
        yield return path;
    }
}

Now that the helpers are described, it's time to get back to the material generation code. When the user picks "Create Materials In Folder", the code will take the user's current selection, finds any "leaf" folders, and then create materials for the textures in that folder. In the code, this is implemented:

[MenuItem("Tools/Art Pipeline/Create Materials In Folder")]
public static void MakeMaterialsFromFolder()
{
    var selection = EditorUtilities.GetValidatedSelection("making materials");
    foreach (var item in selection)
    {
        foreach(var directoryPath in EditorUtilities.GetLeafDirectories(AssetDatabase.GetAssetPath(item)))
        {
            ProcessFilesInDirectory(directoryPath);
        }
    }
}

As you can see, the helper methods are used immediately in the code, to get the selection and to find leaf directories. The method called for each leaf directory, ProcessFilesInDirectoryis one of several (private) methods that are in this same class. It is focused on the file manipulation aspect of the process: it finds which textures are in the folder, and isolates just the textures that are the "albedo" (aka. diffuse) for the material. This is done by assuming a convention: textures that end with "_n" are normal maps, and textures that end with "_o" are ambient occlusion maps. Once it's isolated the textures, it then passes the texture, normal, and ambient occlusion to another (private) method called UpsertMaterialsFromAssets that will be responsible for actually creating or modifying the asset.

I'll include the code for ProcessFilesInDirectoryhere, for those that are curious, but frankly, it's not very well written, so it may not be clear what exactly each line is doing. (One of the factors that interferes with the clarity is that it has to switch back and forth between file paths and asset paths, since Unity does not provide any way of enumerating the individual assets inside a single folder.)

private static void ProcessFilesInDirectory(string directoryPath)
{
    var dir = Path.Combine(Application.dataPath.Replace("Assets", ""), directoryPath);
   var files = Directory.GetFiles(dir).Where(x => x.EndsWith(".png"));
 
    files = files.Select(x => x.Replace(Application.dataPath, Path.GetFileName(Application.dataPath)).Replace("\\", "/"));
 
    foreach (var diffusePath in files.Where(x => !(x.EndsWith("_o.png") || x.EndsWith("_n.png"))))
    {
        var diffuse = AssetDatabase.LoadAssetAtPath(diffusePath, typeof(Texture2D)) as Texture2D;
        if (diffuse == null)
        {
            Debug.LogWarning("Unable to load texture from " + diffusePath);
            return;
        }
        var normal = AssetDatabase.LoadAssetAtPath(diffusePath.Replace(".png", "_n.png"), typeof(Texture2D)) as Texture2D;
        var occlusion = AssetDatabase.LoadAssetAtPath(diffusePath.Replace(".png", "_o.png"), typeof(Texture2D)) as Texture2D;
 
        UpsertMaterialFromAssets(diffusePath.Replace(".png", ".mat"), diffuse, normal, occlusion);
    }
}

UpsertMaterialFromAssets is the method that contains the important logic of this script. It loads the material (if it already exists) or creates a new one (if it doesn't)

 //Find existing asset, if it exists
var asset = AssetDatabase.LoadAssetAtPath<Material>(materialPath);
if (asset == null)
{
    //None exists, make it now and register it with the Asset Database
    asset = new Material(Shader.Find("Standard"));
    AssetDatabase.CreateAsset(asset, materialPath);
}

Then, it configures the material so that it has the correct textures (if they exist) and sets it to Fade Mode (so that it can use partial transparency.) Setting it to Fade Mode is normally a drop-down selection in the UI, but apparently much more complex when done through scripting. (The code for setting fade mode was pulled straight from the linked post- which apparently comes from the code for the Unity's GUI.) Since the Fade Mode code was not my own work, I've kept it as a separate (private) method, and attributed it with a comment. (I expect over time I will end up tweaking it as I become more familiar with shaders.)

    SetMaterialToFadeMode(asset);
    asset.SetTexture("_MainTex", diffuse);
    if (normal != null)
        asset.SetTexture("_BumpMap", normal);
    if (occlusion != null)
        asset.SetTexture("_OcclusionMap", occlusion);
 
}
 
//From discussion on https://forum.unity.com/threads/access-rendering-mode-var-on-standard-shader-via-scripting.287002/#post-1961025
private static void SetMaterialToFadeMode(Material material)
{
    material.SetFloat("_Mode", 2);
    material.SetInt("_SrcBlend", (int)UnityEngine.Rendering.BlendMode.SrcAlpha);
    material.SetInt("_DstBlend", (int)UnityEngine.Rendering.BlendMode.OneMinusSrcAlpha);
    material.SetInt("_ZWrite", 0);
    material.DisableKeyword("_ALPHATEST_ON");
    material.EnableKeyword("_ALPHABLEND_ON");
    material.DisableKeyword("_ALPHAPREMULTIPLY_ON");
    material.renderQueue = 3000;
}

That's it! The code may be long (and more than half of it related to fiddly file-management details) but in the end it does exactly what I need: make new materials and/or modify existing ones to generally set them up the way I would if I were doing so manually through the UI. As always, I'll include the full code below.

#if UNITY_EDITOR
using System.IO;
using System.Linq;
using UnityEditor;
using UnityEngine;
 
public static class MaterialGeneration
{
    [MenuItem("Tools/Art Pipeline/Create Materials In Folder")]
    public static void MakeMaterialsFromFolder()
    {
        var selection = EditorUtilities.GetValidatedSelection("making materials");
        foreach (var item in selection)
        {
            foreach(var directoryPath in EditorUtilities.GetLeafDirectories(AssetDatabase.GetAssetPath(item)))
            {
                ProcessFilesInDirectory(directoryPath);
            }
        }
    }
 
    private static void ProcessFilesInDirectory(string directoryPath)
    {
        var dir = Path.Combine(Application.dataPath.Replace("Assets", ""), directoryPath);
        var files = Directory.GetFiles(dir).Where(x => x.EndsWith(".png"));
 
        files = files.Select(x => x.Replace(Application.dataPath, Path.GetFileName(Application.dataPath)).Replace("\\", "/"));
 
        foreach (var diffusePath in files.Where(x => !(x.EndsWith("_o.png") || x.EndsWith("_n.png"))))
        {
            var diffuse = AssetDatabase.LoadAssetAtPath(diffusePath, typeof(Texture2D)) as Texture2D;
            if (diffuse == null)
            {
                Debug.LogWarning("Unable to load texture from " + diffusePath);
                return;
            }
 
            var normal = AssetDatabase.LoadAssetAtPath(diffusePath.Replace(".png", "_n.png"), typeof(Texture2D)) as Texture2D;
            var occlusion = AssetDatabase.LoadAssetAtPath(diffusePath.Replace(".png", "_o.png"), typeof(Texture2D)) as Texture2D;
 
 
            UpsertMaterialFromAssets(diffusePath.Replace(".png", ".mat"), diffuse, normal, occlusion);
        }
    }
 
    private static void UpsertMaterialFromAssets(string materialPath, Texture2D diffuse, Texture2D normal, Texture2D occlusion)
    {
        //Find existing asset, if it exists
        var asset = AssetDatabase.LoadAssetAtPath<Material>(materialPath);
        if (asset == null)
        {
            //None exists, make it now and register it with the Asset Database
            asset = new Material(Shader.Find("Standard"));
            AssetDatabase.CreateAsset(asset, materialPath);
        }
 
        //Force the material to use certain textures and "Fade Mode" for partial transparency
        SetMaterialToFadeMode(asset);
        asset.SetTexture("_MainTex", diffuse);
        if (normal != null)
            asset.SetTexture("_BumpMap", normal);
        if (occlusion != null)
            asset.SetTexture("_OcclusionMap", occlusion);
    }
 
    //From discussion on https://forum.unity.com/threads/access-rendering-mode-var-on-standard-shader-via-scripting.287002/#post-1961025
    private static void SetMaterialToFadeMode(Material material)
    {
        material.SetFloat("_Mode", 2);
        material.SetInt("_SrcBlend", (int)UnityEngine.Rendering.BlendMode.SrcAlpha);
        material.SetInt("_DstBlend", (int)UnityEngine.Rendering.BlendMode.OneMinusSrcAlpha);
        material.SetInt("_ZWrite", 0);
        material.DisableKeyword("_ALPHATEST_ON");
        material.EnableKeyword("_ALPHABLEND_ON");
        material.DisableKeyword("_ALPHAPREMULTIPLY_ON");
        material.renderQueue = 3000;
    }
}
#endif

#if UNITY_EDITOR
using System.Collections.Generic;
using UnityEditor;
using UnityEngine;
 
public static class EditorUtilities
{
    /// <summary>
    /// Recursively walks all directories in the specified path. Returns the asset path for any folders that do not contain any subfolders (a "leaf" on the tree.)
    /// </summary>
    public static IEnumerable<string> GetLeafDirectories(string path)
    {
        var subfolders = AssetDatabase.GetSubFolders(path);
        if (subfolders.Length > 0)
        {
            foreach (var subfolder in subfolders)
            {
                foreach (var result in GetLeafDirectories(subfolder))
                {
                    yield return result;
                }
            }
        }
        else
        {
            //This is a leaf node, return it here.
            yield return path;
        }
    }
 
    /// <summary>
    /// Yields a list of assets the user has selected in the editor. Items that are not assets will be omitted (and the user will see a warning for each item omitted).
    /// </summary>
    /// <param name="activityDescription">A description of the activity that requires the selection (used in error messages.)</param>
    public static IEnumerable<Object> GetValidatedSelection(string activityDescription)
    {
        if (Selection.objects.Length == 0)
        {
            Debug.LogWarning("Selection required before " + activityDescription);
            yield break;
        }
        foreach (Object o in Selection.objects)
        {
            if (!AssetDatabase.Contains(o))
            {
                Debug.LogWarning("Asset Database does not contain " + o.name);
                continue;
            }
            yield return o;
        }
    }
}
#endif

Conclusion

This wraps up my writing about the art pipeline that I've recently implemented for myself. Overall, I've been tremendously happy to have this automation in place (working on the combat prototype has left me fiddling with a lot of visual details) so this has already started paying back time savings. Writing about the process has been eye-opening (nothing like sharing code with others to help you see it's weaknesses) and, admittedly, a bit grueling (this is more detail than I normally write, as I tried to make it as clear as possible, regardless of experience level.) Hopefully reading this has been valuable to you, and I thank you for joining me on this journey. Next week will be something less technical, as a bit of a break (both for you and for me.) I hope you'll join me then.