Friday, August 19, 2011

XNA Serialization to XML tutorial

Introduction

What is serialization?

Serialization is a semi-automatic process where XNA gets a class that has been marked as [Serializable] and stores/loads it from xml. An awesome idea in theory but with a few shortcomings.

Why these tutorials?

While trying to add serialization to one of my games, I’ve come to notice that there is a very real lack of documentation on XML serialization if you want to do anything more complex than serializing a struct with a few value members inside. When you get to serializing derived classes, dictionaries, trees,… and in general the normal stuff any game needs, there are a lot of hidden pitfalls to be aware of.
It’s a big list of things to go work, but I’m going to share my experiments with serialization and hopefully it will help someone else.

Windows only

This tutorial is aimed only at the windows platform, simply because it’s the only one I’ve got. I promise that if anyone likes them enough to send me a WP7 or an Xbox, I’ll make sure they are compatible. :D

Note*: Check the comments to see the small change to make it work in WP7 and XBox

Serialization example

A look at MSDN serialization

This tutorial is based on the simplest of all options, the default MSDN tutorial for serializing xml, to be found at MSDN: Serialization. The version provided in the download of that article is aimed for the Xbox, which makes it kind of confusing on the windows platform as it uses classes that are not needed, such as the Guide class. If you are working on the Xbox and have doubts, have a look in the book: "XNA Game Studio 4.0 Programming".


My main objection to the example at MSDN: Serialization is that it does NOT tell you where it gets the "device" object, and if you download the code to find out, it uses the "Guide" to get the device in a rather confusing way.


Searching around on how to make it work I found another 3 ways to get the storage device here MSDN : Get StorageDevice. Note that even though they are ALMOST the same as in the MSDN: Serialization example code, the function call varies, not much, but enough to drive one crazy. God bless IntelliSense. If you’ve got more interest, there is a theoretical explanation on how to get a storage device at MSDN : User Storage.

Steps

For both saving and loading the steps are almost the same. Notice most of the work goes into getting the actual file we are going to save to. The function calls to achieve this vary slightly from platform to platform, but we’re going to stick with windows, as the code download at the MSDN: Serialization tutorial has a Xbox version.

  1. User requests to Save / Load
  2. Get the StorageDevice.
    1. Request StorageDevice
    2. Receive StorageDevice
  3. Get a StorageContainer from the device.
    1. Request StorageContainer
    2. Receive StorageContainer
  4. Get the Stream of the file we want to use from the container.
  5. Serialize / Deserialize.
  6. Cleanup.

Code

With that out of the way, let’s head over to the code!
We will be using a very simple data class to serialize in this first tutorial.


Code Snippet
  1. /// <summary>
  2. /// Data class with some basic data
  3. /// for testing.
  4. /// </summary>
  5. [Serializable]
  6. public class Data1
  7. {
  8. public int age;
  9. public string name = "Chuel";
  10. /// <summary>
  11. /// Initializes a new instance of
  12. /// the Data1 class.
  13. /// Required default constructor for
  14. /// serialization.
  15. /// </summary>
  16. public Data1()
  17. {
  18. this.age = 0;
  19. }
  20. /// <summary>
  21. /// Initializes a new instance of
  22. /// the Data1 class.
  23. /// </summary>
  24. /// <param name="age">Age of the chuel.</param>
  25. public Data1(int age)
  26. {
  27. this.age = age;
  28. }
  29. public void Draw(SpriteBatch spriteBatch, SpriteFont font)
  30. {
  31. string text = "Name: " + this.name +
  32. " Age: " + this.age;
  33. spriteBatch.DrawString(
  34. font, text,
  35. new Vector2(10, 10),
  36. Color.White);
  37. }
  38. }


First thing to notice here is that you need to mark with the [Serializable] attribute the class you want to serialize (Not necessary for serializing to XML, check r2d2rigo's comment), there are more options to explore with related attributes and you can find it here. Second, that it’s a public class, as the object you serialize must be public. Third, only public properties or fields will be serialized (age and name).


And here is how we load / save the data.

Code Snippet
  1. /// <summary>
  2. /// This is a game component that
  3. /// implements IUpdateable.
  4. /// It serializes data in a simple way,
  5. /// no preprocesing or special content.
  6. /// No blocking when doing the async. calls.
  7. /// </summary>
  8. public class Mode2 :
  9. Microsoft.Xna.Framework.DrawableGameComponent
  10. {
  11. public Data1 data1;
  12. private SpriteBatch spriteBatch;
  13. // Keyboard polling helpers
  14. private KeyboardState currentState;
  15. private KeyboardState previousState;
  16. // Action to do
  17. private enum State {
  18. Innactive, RequestDevice,
  19. GetDevice, GetContainer,
  20. SaveLoad };
  21. private State state;
  22. private enum Action { None, Save, Load };
  23. private Action action;
  24. // Needed variables
  25. IAsyncResult result = null;
  26. StorageDevice device = null;
  27. StorageContainer container = null;
  28. public Mode2(Game game)
  29. : base(game)
  30. {
  31. state = State.Innactive;
  32. action = Action.None;
  33. }
  34. public override void Initialize()
  35. {
  36. base.Initialize();
  37. this.spriteBatch =
  38. new SpriteBatch(Game.GraphicsDevice);
  39. this.data1 = new Data1();
  40. }
  41. public override void Update(GameTime gameTime)
  42. {
  43. // Key checking
  44. previousState = currentState;
  45. currentState = Keyboard.GetState();
  46. // STEP 1: User requests save / load
  47. // When we press the "s" key
  48. if (currentState.IsKeyUp(Keys.S) &&
  49. previousState.IsKeyDown(Keys.S))
  50. {
  51. // Request to save
  52. if (state == State.Innactive &&
  53. action == Action.None)
  54. {
  55. this.state = State.RequestDevice;
  56. this.action = Action.Save;
  57. }
  58. }
  59. // When we press the "l" key
  60. if (currentState.IsKeyUp(Keys.L) &&
  61. previousState.IsKeyDown(Keys.L))
  62. {
  63. // Request to save
  64. if (state == State.Innactive &&
  65. action == Action.None)
  66. {
  67. this.state = State.RequestDevice;
  68. this.action = Action.Load;
  69. }
  70. }
  71. // Do the actual save or load
  72. if (this.action == Action.Save &&
  73. this.state != State.Innactive)
  74. {
  75. this.Save();
  76. }
  77. if (this.action == Action.Load &&
  78. this.state != State.Innactive)
  79. {
  80. this.Load();
  81. }
  82. // Make the age change so we can see
  83. // stuff change with the save / load
  84. if (Mouse.GetState().LeftButton ==
  85. ButtonState.Pressed)
  86. {
  87. this.data1.age += 1;
  88. }
  89. base.Update(gameTime);
  90. }
  91. /// <summary>
  92. /// Function where we do all the saving
  93. /// </summary>
  94. private void Save()
  95. {
  96. switch (this.state)
  97. {
  98. case State.RequestDevice:
  99. {
  100. // STEP 2.a: Request the device
  101. result =
  102. StorageDevice.BeginShowSelector(
  103. PlayerIndex.One,
  104. null,
  105. null);
  106. this.state = State.GetDevice;
  107. }
  108. break;
  109. case State.GetDevice:
  110. {
  111. // If the device is ready
  112. if (result.IsCompleted)
  113. {
  114. // STEP 2.b: Recieve the device
  115. device =
  116. StorageDevice.EndShowSelector(result);
  117. // STEP 3.a: Request the container
  118. result.AsyncWaitHandle.Close();
  119. result = null;
  120. result = device.BeginOpenContainer(
  121. "StorageDemo",
  122. null,
  123. null);
  124. this.state = State.GetContainer;
  125. }
  126. }
  127. break;
  128. case State.GetContainer:
  129. {
  130. // If the container is ready
  131. if (result.IsCompleted)
  132. {
  133. // STEP 3.b: Recieve the container
  134. container = device.EndOpenContainer(result);
  135. result.AsyncWaitHandle.Close();
  136. this.state = State.SaveLoad;
  137. }
  138. }
  139. break;
  140. case State.SaveLoad:
  141. {
  142. string filename = "savegame.sav";
  143. // Check to see whether
  144. // the file exists.
  145. if (container.FileExists(filename))
  146. {
  147. // Delete it so that we
  148. // can create one fresh.
  149. container.DeleteFile(filename);
  150. }
  151. // Create the file.
  152. Stream stream =
  153. container.CreateFile(filename);
  154. // Convert the object to XML data
  155. // and put it in the stream.
  156. XmlSerializer serializer =
  157. new XmlSerializer(typeof(Data1));
  158. serializer.Serialize(stream, data1);
  159. // Close the file.
  160. stream.Close();
  161. // Dispose the container, to
  162. // commit changes.
  163. container.Dispose();
  164. state = State.Innactive;
  165. action = Action.None;
  166. }
  167. break;
  168. }
  169. }
  170. /// <summary>
  171. /// Function where we do all the loading
  172. /// </summary>
  173. private void Load()
  174. {
  175. switch (this.state)
  176. {
  177. case State.RequestDevice:
  178. {
  179. // STEP 2.a: Request the device
  180. result =
  181. StorageDevice.BeginShowSelector(
  182. PlayerIndex.One,
  183. null,
  184. null);
  185. this.state = State.GetDevice;
  186. }
  187. break;
  188. case State.GetDevice:
  189. {
  190. // If the device is ready
  191. if (result.IsCompleted)
  192. {
  193. // STEP 2.b: Recieve the device
  194. device =
  195. StorageDevice.EndShowSelector(result);
  196. // STEP 3.a: Request the container
  197. result.AsyncWaitHandle.Close();
  198. result = null;
  199. result =
  200. device.BeginOpenContainer(
  201. "StorageDemo",
  202. null,
  203. null);
  204. this.state = State.GetContainer;
  205. }
  206. }
  207. break;
  208. case State.GetContainer:
  209. {
  210. // If the container is ready
  211. if (result.IsCompleted)
  212. {
  213. // STEP 3.b: Recieve the container
  214. container =
  215. device.EndOpenContainer(result);
  216. result.AsyncWaitHandle.Close();
  217. this.state = State.SaveLoad;
  218. }
  219. }
  220. break;
  221. case State.SaveLoad:
  222. {
  223. string filename = "savegame.sav";
  224. // Check to see whether the save exists.
  225. if (!container.FileExists(filename))
  226. {
  227. // If not, dispose of the
  228. // container and return.
  229. container.Dispose();
  230. return;
  231. }
  232. // Create the file.
  233. Stream stream =
  234. container.OpenFile(
  235. filename,
  236. FileMode.Open);
  237. // Convert the object to XML
  238. // data and put it in the stream.
  239. XmlSerializer serializer =
  240. new XmlSerializer(typeof(Data1));
  241. data1 =
  242. (Data1)serializer.Deserialize(stream);
  243. // Close the file.
  244. stream.Close();
  245. // Dispose the container, to commit changes.
  246. container.Dispose();
  247. state = State.Innactive;
  248. action = Action.None;
  249. }
  250. break;
  251. }
  252. }
  253. public override void Draw(GameTime gameTime)
  254. {
  255. base.Draw(gameTime);
  256. this.spriteBatch.Begin();
  257. this.data1.Draw(
  258. this.spriteBatch,
  259. Game.Content.Load<SpriteFont>("Font"));
  260. this.spriteBatch.End();
  261. }
  262. }

We need to get 2 objects with asynchronous calls, the device and the container. This means we need to be careful not to block the game while we wait for the calls to return, and that’s why the code looks like a magical rainbow of cases, because the Update() function might be called lots of times while we wait and we don’t want strange stuff to happen.


We create a data object with just a name and an age, when we press the left button of the mouse the age increments, press “s” to save and “l” to load the last saved data.


Following the comments in the code should give a pretty clear idea of what is going on, but we’ll go over it none the less, I’ll ignore the support code not directly related with serialization, but if there are any doubts just ask in the comments. There is some redundant code but I haven’t shortened to make it as simple as possible to follow the logic.


Both save and load work very similar. First thing we need to do is get a storage device. This requires using an asynchronous call, so we request it (2.a) by calling BeginShowSelector and when it’s ready result.IsCompleted will return true. Notice that in windows we pass PlayerIndex.One as the parameter.

We then receive the device (2.b) and request the container from the device (3.a) with BeginContainer, where we pass the name of the “container” to use, in this case, the folder StorageDemo.


We finally we receive the container (3.b) and we’re ready to go! No more magical asynchronous stuff.


The last part is where the actual serializing takes place. For saving we get ourselves a file to write on (4) and a serializer for the type of data we want to serialize (Data1 in this case), and then just serialize it into the file (5) and tidy up (6). For loading we check the file exists (4) and then open it and deserialize into our data object (5) and clean up (6).

The generated XML looks like this:

Code Snippet
  1. <?xml version="1.0"?>
  2. <Data1 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns:xsd="http://www.w3.org/2001/XMLSchema">
  3. <age>43</age>
  4. <name>Chuel</name>
  5. </Data1>


A last note on how the system is set up. The Mode2 class is a DrawableGameComponent so that the XNA framework takes care of everything (updating and drawing and so on), and in our game class we only need to add one line. Has nothing to do with serializing.

Code Snippet
  1. public Game1()
  2. {
  3. graphics = new GraphicsDeviceManager(this);
  4. Content.RootDirectory = "Content";
  5. this.Components.Add(new Mode2(this));
  6. }

Notes

Data that XNA serializes

XNA doesn’t serialize everything that there is in the class, but the documentation is not too specific on what’s game besides “public properties and fields”. So I went ahead a did a little testing. Here are the results of testing, plus what I found searching around the net.

  • Serializes public properties, not private or protected.
  • Serializes public fields with both get and set, not partially implemented fields.
  • Arrays of values or of objects work fine.
  • Lists<> of values or of objects work fine. Tough not generic List<T>
  • Cannot serialize ArrayList.
  • Cannot serialize an enumeration of unsigned long with any value larger than 9,223,372,036,854,775,807.
  • Typical XNA classes like Rectangle or Vector work just fine with the serialization.
  • On serializing references, I found this bit from the MSDN forum:

    “It will serialize references, but always as a copy. Basically what this means is that each time the serializer hits an object that has a reference to another serializable object exposed as a public property or field, it will write out the all of the public properties/fields of the object that reference points to. This is a problem if you have the same object referenced multiple times by something you're serializing. For instance...say you had two MeleeAttack's referencing the same PhysicsObject. After deserialization, you'd end up with your MeleeAttack's having references to different PhysicsObject's with identical properties. I used to get around this by giving certain types of objects a unique integer ID, and then having the objects serialize the ID rather than the reference. Then after deserialization I'd have another initialization step where I'd use the ID to look up the corresponding object and grab a reference.”
  • From textures it only saves the name.

Leads for more complex serialization magic

For elements that we read from Xml that are not in the object, we can use the XmlAnyElementAttribute and XmlAnyAttributeAttribute attributes. Useful for compatibility with older versions of our code. Check it here.

To ignore a field we can use the XmlIgnoreAttribute. Check it here.

You can override the serialization of any set of objects by creating one of the appropriate attributes and adding it to an instance of the XmlAttributes class. More complex stuff that can be used to serialize things that come from dlls, or to control how to serialize your own data. Check it here.

You can use the IXmlSerializable interface to provide custom formatting for Xml serialization and deserialization. Check it here.

4 Quick rules

To end, 4 quick reference rules to keep in mind for serializing:

  • Note #1: You need to mark the class you want to serialize with the [Serializable] attribute.
  • Note #2: Serialized classes must have a default constructor.
  • Note #3: Serialization only converts the public properties and fields
  • Note #3: The actual object you serialize must be public.

Conclusion

The serialization mechanism is awesome as in it’s automatic for the classes marked, but it’s lack of handling references correctly makes it useless for saving a big “chunk” of game data (like a whole level) as it doesn’t avoid the work of having to set up a ID system to take care of the references. It is best suited for things as saving the player preferences, or the control settings or similar.

4 comments:

  1. Quick correction: you don't need to mark your class as [Serializable], since that only works for binary serialization; and the code should work for WP7 and Xbox too with the minor addition of System.Xml.Serialization assembly.

    Nice tutorial!

    ReplyDelete
  2. @r2d2rigo Thank you for that :D So, no need to mark as [Serializable] for serializing for XML?

    ReplyDelete
  3. @Jaime: nope! That's only required for BinarySerializer, since XmlSerializer does everything at runtime using reflection. And Xbox and WP7 can only serialize with XMLs!

    ReplyDelete
  4. A big thanks... it relieved me after hours of frustration.

    ReplyDelete