SDL_RWops for Custom Storage Implementations

I was reading an older post on Julia Evans’ blog about sharing things you’ve learned to help others, and I realized I worked on something earlier this year that was a bit niche and I couldn’t find much other writing about: loading data into SDL from something that is not a plain file handle or path on the filesystem. I got pretty lucky and figured it out after a few hours of hacking on it, but I figured I’d share since it was rather abstract. Maybe I’ll be able to save you a few minutes.

Why Not Load from a File Path?

Any time I do something vaguely off the beaten path, I hear a chorus in my head of the best engineers I’ve known in my life ask “why?” There are a few reasons why the most obvious and simple choice (loading assets from a file path) may not meet your requirements:

The latter was my use-case, though I’ll also mention that this was not a professional project, so “it sounds like an interesting problem” was probably 50% of the reason I went this direction.

For the project I was working on (a game engine), I wanted to use bundled assets as a singular archive with a format I controlled, with the hope that cross-platform support would be obvious, installation/copying would be straightforward, and there would be a very obvious artifact that could be published. This worked great, and with a little bit of effort, I was able to start packing and loading scripts from this format, eschewing many file location headaches along the way. For the purposes of this post, I won’t go into any detail into my specific implementation, so I’ll handwave the details away as “my custom implementation” for now.

At least it was going great, up until I needed to start loading SDL_Texture data. IMG_LoadTexture (where most documentation/guides will point you) obviously doesn’t fit the bill since it takes a filesystem path. The docs are kind, however, and hint towards what to use if this is insufficient:

There is a separate function to read files from an SDL_RWops, if you need an i/o abstraction to provide data from anywhere instead of a simple filesystem read; that function is IMG_LoadTexture_RW().

Most literature I found on SDL_RWops used it for handing SDL file handles (not paths) or in-memory structures, but it still seemed like the right abstraction to pursue.

What is SDL_RWops?

Before we dive into the specifics of functions that build on top of SDL_RWops, we need to work backwards and understand what an “RWops” is conceptually.

At last, clarity!

/**
 * # CategoryRWOPS
 *
 * This file provides a general interface for SDL to read and write data
 * streams. It can easily be extended to files, memory, etc.
 */

This, in tandem with the struct defined in the header, tells us we can build an SDL_RWops struct, set some fields, and if all goes well, we can pass this struct to the RW family of functions rather than the more common SDL APIs that use file paths. You can think of this is a generic “IO” interface. SDL provides some default implementations (e.g. in-memory, from a file handle, etc.) or you can fill in the implementation with something custom. As long as the interface is met, these structures can be used interchangeably, regardless of the underlying storage being used.

Initializing an SDL_RWops Struct

Let’s ask for an RWops:

SDL_RWops *ops = SDL_AllocRW();

Next, we need to assign a “type” to this struct to tell SDL which implementation is being used. If you spend enough time in the header looking at the function definitions and the hidden union (this is the private data specific the underlying implementation), you’ll notice that most of the options for type have an implmentation already or they have a name that clearly rules them out e.g. androidio. There’s only one choice that makes any sense: SDL_RWOPS_UNKNOWN. If you look at hidden.unknown, it just stores a couple of void *, which should be plenty flexible and generic for our customization. EZPZ:

ops->type = SDL_RWOPS_UNKNOWN;
ops->hidden.unknown.data1 = my_custom_file_handle;

Obviously, set data1 to a pointer that is relevant to your backing storage. With this in place, all we need to do now is define or stub some basic implementations for this interface.

Implementing the Interface

Now you need to define functions according to the header for:

I’m not going to review every single function that needs to be implemented, but I’ll show two examples. Firstly, let’s do size since it’s relatively simple and self contained:

static Sint64 custom_rwops_size(SDL_RWops *context) {
	my_custom_file_handle_t *handle = (my_custom_file_handle_t *) context->hidden.unknown.data1;
	return (Sint64) my_custom_file_size(handle);
}

This covers the basic flow of every operation you’ll implement, where you’ll start by casting your data out of the private unknown.data1 field, and then you’ll build on top of your own implementation to meet the interface. As with all C code, be mindful of allocations and leaked resources. Once you’ve written the function, make sure that your struct initialization assigns it to the corresponding field:

ops->size = custom_rwops_size;

Sometimes we don’t want support something at all! Stub it:

static size_t custom_rwops_write(SDL_RWops *context, const void *ptr, size_t size, size_t num) {
	MY_PANIC("Sorry Dave, I can't let you do that");
	return 0;
}

…And that’s it. Once you’ve defined (and assigned) your implementations, you can drop in your RWops in any _RW function:

SDL_Texture *texture = IMG_LoadTexture_RW(renderer, ops, 1);

Conclusion

Overall, the SDL_RWops API is quite flexible, and once you find the unknown type in the headers, it’s very straightforward to use! It’s also a nice example of a generic “interface” implementation in C, and it was mercifully straightforward to set up. Given that there are two active versions of SDL being used in the wild and that this is an open-source project, I definitely can forgive the maintainers for requiring a little bit of digging for a relativley niche feature.

Happy hacking! o7