So Much Progress!

Checking out Earth in the solar system builder

Manually loading member data

It may not seem like much, but the filesystem replacement functions are going really well. I can now load, edit,  and save the solar system files in their native format and without regard of endianness (In the original code, Apple and Atari had special functions to byte swap integers before they were written to disk.). I decided to to write the code so that each struct member is loaded individually. This means that I don't need to rely on how big an integer is anymore. The real success was when I loaded and saved the solar system file and did a binary compare and they both came out the same. Then I added a system, deleted it and saved it and it was still the same... 100% compatibility!

Captain list

I just completed work on the code to load the captains. I'll have to work on the code to save them tomorrow. After that I'm going to do the ships and the enemy races. The underpinnings are working well with the HD graphics library. Even though these graphics look pixelated, that's because I'm using the original assets zoomed 400%. When the updated graphics come along, you will start to see things begin to shine.






Another thing I came across was an interesting piece of code that will not compile anymore. You can tell it was written in a time when you could overwrite memory with no protection. It was the code that replaced a tilde (~) with a string you passed it. The original code looked like this...

cs = strchr(ts,'~');

if(cs != NULL) {
  *cs = 0;
  strcpy(bs,ts);
  *cs = '~';
  cs++;
  strcat(bs,s);
  strcat(bs,cs);
  }

else strcpy(bs,ts); 

You just can't arbitrarily change data at a pointer that's pointing to... something anymore. [*cs = 0;] - Windows gets super upset when you try. It was fixed with a little pointer math like such.

cs = strchr(ts, '~');
if (cs != NULL) {
   n = cs - ts;        //n = characters from ts to cs
   memcpy(bs, ts, n);  //copy up to cs into bs
   bs[n] = '\0';       //terminate the string
   cs++;               //push the pointer up one to skip the tilde
   strcat(bs, s);      //cat "file" to bs
   strcat(bs, cs);     //cat the rest of the string to bs.
}
else strcpy(bs, ts);

You have to be polite as ask to do things with memory as opposed to plopping a pointer anywhere you want.

Anywho, Here's a few final screenshots of progress.

Sol System - HD circle on radar screen

Captain Dossier

Integers Strike Again

Missions can now load and be viewable

Slowly the filesystem functions is being replaced. I can now read the mission and fleet commander files. In fact, I'm using one of the original files used by the original developer to test the game. (His commander is named after him) The time stamp when he last played the game was 7:48pm Dec, 18th 1992.


I have now run into a nasty issue I thought I wouldn't run into, but it's back again. The game was written on an compiler that used 2 byte integers. Systems nowadays have integers that are 4 bytes, or in the case of 64 bit Linux and MacOS systems, 8 bytes. Normally this wouldn't be a problem, but the way the game loads data objects from disk is   VERY byte dependent. The system just loads up the data as a block of serialized bytes and lays them directly into a struct from top to bottom. So you can imagine what happens when the data was saved with 2 byte data members and then loaded and put into a struct that has 4 byte data members.

int was replaced with int16_t

I had to change the int types to int16_t to get them to fit, but I'm thinking this is going to cause casting issues down the line. I'm already having issues with capturing the saved outpost data into the linked list properly. I'll battle that tomorrow. If worst comes to worst, I'll have to manually load each struct member individually. I'm sure the old way was optimized for speed and size. Computers have plenty of that to spare nowadays. Allegro can load integers of various sizes and endianness individually, so I can probably leverage that too.

In for a Penny, In for a Pound

Today was the day I started to replace the filesystem functions in the game. This means that any reference to  fopen(), fread(), fwrite(), fclose() and other file functions are out with Allegro replacements put in.

I already started with replacing FileFindFirst() and FileFindNext() which were wrappers to the long dead _dos_findfirst()  and _dos_findnext() functions (Well, not really, you can still find those in DGJPP). The Windows equivalent of _findfirst() and _findnext() were not compatible with the DOS version. On top of this all these functions are only found in the Microsoft ecosystem.

Finally, they were all underscore functions and have to go anyway.

After rewriting the functions, I now discover I can't mix and match "normal" filestream functions with Allegros, so they all have to go. Looks like I know my next project now!

In other news, My artist, Cody, has been hard at work making my original graphics pretty! The original game used three skin tones (pink, yellow, and brown) to donate races. Now things are much more racially diverse thanks to an idea he had.

The uniforms are getting a makeover an now I have battle damage! Thanks Cody!

HD Nermal

I normally don't work on Saturdays, but I had an itching to get all the junk out of Nermal and make it completely HD capable. I pretty much got all the drawing functions done and completed BitBlit().

BitBlit() is the function that copied bitmaps from one buffer to another. I only implemented B_COPY mode which is a simple bitmap copy. Now All 7 modes are implemented (B_OR, B_AND, B_XOR, B_NOT, B_REV, and B_CUT) Each mode copies the colors for the source and pastes them preforming a bitwise operation on the detestation. With only 16 colors this can be dicy, but with 24 bit color, you just preform the operation on each color channel.

HD test of Nermal primitives

Because the resolution is higher, but still made up of a 320x200 "grid" I was wondering if I should make my lines 4x as thick... I decided not to for now, so I can make super thin lines, rectangles, and circles.

My big challenge next is to get rid of _findfirst() and _findnext() functions. These directory search functions are only found in the Microsoft ecosystem, and Allegro uses an iterator and callback function to search directories instead. I will need to create an equivalent function in Allegro like I did for kbhit() and getch(). That's going to be tricky.

I also need to remove the rest of the legacy filesystem functions and get those into Allegro as well. After that... I'll be attacking the 1Ch thread one last time and making it stable.

Copyright traps

So in the intervening week I have hired an artist to help with some of the game. Even though I can passably draw, I am not commercial level good. One of the issues I have run into is that I can't draw attractive looking human faces. When I was drawing the characters, I was relying on a cute little face maker I found online to help with the heads. It was fun and free and made great looking heads to put on my body...

The website was http://avatarmaker.com/

It bugged me that there was no terms of service on the website. It was just "Make a head" and then "download SVG" when done. Now I know that art does not magically appear out of thin air, and artists like to eat. So this had to come from somewhere. I couldn't believe that this was being released out of the kindness of someone's heart. I decided to see if I could maybe contact the author of the site. First thing was to do a whois to see who owned the  avatarmaker.com domain.

Well, that was fruitless. They have protected their contact info behind a spam shield. This was odd though, because this is a service used mostly by companies. Ok, in this case I decided to dig into the code of the face maker itself.

BINGO! found it, turns out that the program is a demo for SVGAvatars. However, it seems really strange to have the info buried in one of the page's javascript dependencies. I mean, why wasn't there a banner or something letting a prospective user know that this was a demo for a commercial product? It seemed more like a watermark than something you would use to track down the original developer. Following the link the svgavatars.com, things went down an even stranger path...

Turns out Svgavatar is a web plugin that's for sale for $30... but the terms are weird. It says that if I want to use the plugin beyond the scope the the price tag that I need an "extended license", but was unable to find any contact or purchase info on what this included. There was a link to the developer so, as any rational person, decided to try and contact this person directly.

It turns out that in order for you to contact Mr. or Ms. DeeThemes, you have to have a CodeCanyon account, which for $22 a month, which  "allows you to download free themes, plugins and other media"

I was increasingly believing that this avatar maker was a copyright trap.

There has been a growing trend of copyright holders putting "freebies" content on the internet for people to download and put into their works. The terms are usually hidden but has some banner about it being free and all. If you use the content in your work, the trap is sprung. A lawyer will spring into action asking to settle for a few grand for copyright infringement. Because of the small scale, it's profitable to the lawyer to do this for little work. Then the content creator gets everything above the lawyers frees, which is about a billiable hour.

Newp! Not having any of that! I decided to ditch the heads and to have someone give me hand. Wasn't worth the possible infringement, not matter how "Free" it was.

More High Resolution

It may look the same, but it's actually 1280x800

As the framework for the HD comes onto place, I need to actually give the game high resolution assets to work with.. For now to get things to function I've been blowing up all the old 320x200 to 400% to work as templates. I also have brought on an artist (Yay!) to help with some of the work too. I also figured out how to bring the brightness up on my tinting, but not the contrast. That means that when I brighten things, my blacks become greys. I'm not quite sure just yet where I'm going to be taking the graphics so I may implement the tinting differently.

So, all in all, it's still getting the main engine to take the HD changes, replacing the legacy functions I don't need, and cleaning up the ugly hacks I put in to make the game run.

Swiss flag in ROE

Now that I'm getting a good look at the graphics as I scale them up, I have to keep a running list of art that needs to be changed for legal reasons. Here's one that's interesting. The Swiss flag is used often used for neutral territories and the inverted flag, the Red Cross, is used as a medical help symbol. However, the Red Cross is a trademarked symbol and has gone after video game companies in the past when the Red Cross was used on health kits and such.When I saw that ROE uses the cross for neutral outposts I was all "Whelp! Gotta get rid of that too!" Turns out that's fine, and the Swiss are more than happy for people to use their flag. The Red Cross, however are the guys are are bent out of shape when the Red Cross or the Red Crescent is used.

Old vs new Halo health packs

I'll still have to get rid of the Teenage Mutant Ninja Turtles though...

Layers, Like an Ogre

Above the Nermal graphic library, (which is sitting on Allegro), there is rules0.c that holds many common functions for the game. This is where my errant flashing buttons are that's giving me my grief. Turns out the reason why I was having a problem pinning it down was because the functions were formatted in such a way that Intellisense couldn't see them.

Normally in C, to define a function, you create a prototype at the top. This is so you don't surprise the compiler when you start compiling functions later on with undefined functions inside. The caveat is that if you define a function before it's used, you don't need a prototype. It's always good practice to prototype your function though, so that you don't put the cart before the horse.

Rules of engagement does not use prototypes. There is a proto.c file where they are supped to go, but it was not current and all the functions are defined before use in the code itself. However, in rules0.c the definitions were valid, but oddly formed. In fact, I didn't even know you could do it like this

Here's an example.

int Restrict(val, l, u)
int val, l, u;
{
    if (val < l) return(l);
    if (val > u) return(u);
    return(val);
}

The top two lines should be combined like so .

int Restrict(int val, int l, int u)

This makes the definition more pototype-y. Because the variables were not typed in the first example, Intellisense didn't register it as a function and was "invisible" to my IDE. (They compiled fine though) My job today was to go though rules0.c and correct all the definitions. While I was at it I found the FlashButton() function that was calling ColorMask() that was giving me all the grief. As opposed to relying on an old legacy color function, I can rewrite FlashButton() to flash selected buttons in a more modern way and not relying on legacy color masks at all.

While I was at it I also updated all the documentation in rules0.c for Doxygen.Now I shouldn't have any lost functions anymore. I didn't do any of the variable documentation yet, but now I can modernize some of the more "middleware" functions.

Rules0.c is documented now

Color Condundrum

Today was so much more productive. I discovered that the three to five second delay I was having with my screen updates. It was due to a really nasty hack I put into Nermal to make it function. I decided to get that working once and for all. I don't have a solution yet, but I'm working on it.

Rules of  Engagement originally ran in 4 bit color. This gives you 2⁴ or 16 colors to play with. With so little colors, you did not have a "transparency". ROE gets around this by assigning black as a "mask". This means that you can have some functions that alter colors, except for black. This becomes useful later.

I'm trying to duplicate this functionality in Allegro 5 with 24 bit color. (If you are counting 2²⁴ is 16,777,216 colors to work with. Considerably more!) Besides the boost in colors, I also have variable transparencies as well.

 My grief comes down to a function called DoColorMask(). In the original game, the function would copy a bitmap from a source location to a destination. While the the bitmap was pixel-for-pixel being copied over, it would check the color of the pixel being copied. If the pixel isn't black, then change the pixel color to a color you would give it. This means that if you have a solid red colored button with black text, you can use the function to make the button white with black text. This function is often used to make buttons "flash"

The upshot is this is a poor-man's tint function.

Allegro has a comparable function called  al_draw_tinted_bitmap_region(). It's promising, except there are issues I'm not sure how to resolve yet.

Full, Red, Green, and Blue Channels

The first thing I did was try a few color tinting experiments. I took a standard colorbar pattern and decided to see what happened when I ramped the tint to 100% in a particular color channel. In the pictures you can see in the upper left  the "normal" color bars. The upper right is red channel, lower left is green channel and lower right is blue channel. The problem with this approach is that it will "block" all colors that are not in that channel. For example, the blue in the red channel is black because... well, there is no blue in the red channel, only red.

Using alpha to "Push" a channel color

My second experiment was to overlay  the altered color channel back into the normal channel with a 50% opacity. That looked a little better and I can now "push" a color towards the color tint I want. However a problem with this is I can't alter the luminosity. In other words, I can't make a color brighter then the original color. (I can't turn red into pink, or white, just into a mix of blue or green).

Can't go above full brightness

I tried a few luminosity experiments, but that just kind of confirmed what I thought. I have the ability to make the bitmaps darker, but I can't monkey with the gamma the way I want with the tools I have. It's not horrible. It just means I need to make better tools.

Gimps "Colorize" in the blue channel.

The solution I actually need is use something like Gimp's "Colorize" function. How that works is that you give it a picture, and it will strip everything out but the luminosity channel (Makes it black and white). Then it applies a color tint to that channel. From there you adjust three sliders for Hue, Saturation, and Lightness. I don't think there is an easy way to do this in Allegro, well, short of brute-forcing it on a pixel-by-pixel basis, and that's what I'm trying to get away from. I'll be doing more colorspace experiments tomorrow.

I was told there would be be days like this....

Last week during my excursion into art I decided to move let my circadian rhythm slip to something like 3rd shift. This is a horrifically bad habit of mine and I always forget the consequences of this action when I have to stay up all day. I just wind up a big lump of feeling awful for hours with little, if any, productivity.

I got nothing done today.

Well, I'm kind of lying. I fixed up the mouse so that it works in both windowed and fullscreen mode. So input is working in HD which is nice I guess. It was munging about 20 lines of code to get the offsets and scale correct. I'm pretty sure a "real" programmer probably would of knocked out the issue in about 2 minutes. I'm using "real" in heavy air quotes, mind you. I have code to work, and my goal is to have a functional application. Sounds like I'm filling the shoes of a programmer to me...

One upon a time, many moons ago I was a database programmer for the local municipalitiy's police force. I lasted about 6 months before I realized that database architecture was not my bag. My job was to take flat form data from a VAX machine, and then write parsers that would convert them into an rational "insert" statement for PostgreSQL. My weapon was Ruby. At the time it was a rather unknown language from Japan that had aspiration to unseat Perl as the object orientated scripting language for the world.  (This was before "Rails" was a thing.)

The problems I had was few-fold. The biggest one was I didn't know any object orientated paradigms at all. When I did research on it, I eschewed the whole philosophy. I remember reading an introduction to OO, which listed 10 things to never do while writing a program.

"1) No Globals."

Upon reading that, I pretty much thought OO was garbage and didn't want to do anything with it. I saw it as corrupting simple programming, making it unnecessarily complex and assigning silly names for stuff that was already invented. It was a "method" now, not a "function". Oh that's called a "member field" and not a "variable". Even something as simple was "what is a class" was answered in terms of  "it's whatever you want it to be". It seemed like new-age trash.

I'm a little wiser now.

I like working with ROE because it calls back to my old programming habits. It's control flow is awful. The game literally doesn't have a "main loop", just a bunch of state machines handled by globals that switch the program pointer to different parts of code like a track switcher steering a train down a path of multiple tracks. It's complicated to follow and it's hard.

After dolling up my mouse code, it works good enough, but not really how I want to see it function. When the mouse goes "out of bounds" it hides in strange corners of the screen. But, really, I'm not interested in out of bounds mouse locations anyway. (At least not right now)

My new issue is that when the screen updates it doing it about a frame every five seconds. So when I pushed the game to 4x the resolution, there is something just eating CPU cycles now and I didn't want to tear anything apart to fix it today. I'm pretty sure it's my new timer code but.... ehhhhhh.

Obviously, it's going to need to get done, or else I simply will not have a game and this whole endeavor would be a huge waste of time and money.. Here's hoping tomorrow will be better.

(It most likely will be after I get an actual 8 hours of sleep)

Tunneling

One of the things they don't tell you about independent development is that the "Indie" bit can also stand for "individual". It can get pretty lonely. I've been rolling with the project for about three months now and I'm quite happy with the progress, In fact, moving on to the high resolution conversion has made me gain a new appreciation for "art" side of things I can do. I mean, I didn't write the opening theme for the game, but I was proud that I was able to re-sequence it. The graphics are not my design, but updating them to HD has been an interesting experience. Now I'm stuck with a big laundry list of things to do and it seems like every goal is just moving the flag a little bit and I realize I'm pretty much hunkered down for the long haul.

I've beginning to network with other indie developers because this has been a solitary endeavor as of late. I tossed out this devblog and I got a spike in readership, so that's cool.

One of the things I realized is that I'm in a position where if a task needs to be done. I'm the one to do it. There is no one else to pawn something on to so if I don't know how to do it, well, it's time to learn then.  Remember I said the heads are temporary for my paper dolls. That's because the art isn't mine and I'm putting in placeholders. I was hoping I could get away with them, but I can't. The bodies are fine and I draw them up with reference art, but why it looks cool is because I draw my charterers REALLY BIG and then scale them down. In the process all by awful lines and bad art is aliased to a blur.

It works though.

However, I decided I need to do the heads properly, and that's a detour that I'm going to take now other than later. I have to be really careful here as it's super easy to hang up on art and not get going to the next thing. I'm going to give myself a week to get the paperdolls all in order. This means heads. damage, rank, and everything. This means I need to pick up the pace a little. New goals are fun!

I see why Bob Ross didn't like drawing people

I have the Admiral done. It's getting kind of monotonous drawing people. My paperdolls not "done" done. In fact, the heads are temporary. I also haven't drawn any of the battle scars or anything yet/ These do, however show off the direction I'm going in. My last goal is to do the outpost(?) staff lady. She's just wearing a jumpsuit, with no swappable heads. She has a background too. When that is done I will be going back into "Font mode" and will be fixing up the "gprint" font. It's not a fixed width font so I'll be grabbing selecting the proper point pitch and then getting each letter to fit on an individual basis. It would be kind of awful if after all my work, find that the letters overrun.

Anyways here is some more pretty graphics so you can see how they all fit

The EGA color palette surprisingly works! Can you tell I don't like hands?

Female is 80% there now too.

Female Captain - Old and new

So I have the female captain done now. There are not "done" by any stretch of the imagination. I wil; be doing the Admiral next. After that I have to add some details (ripped battle clothing and then rank on the left and side) and then do the 25 heads. (4 Admiral, 16 male and 8 female). Looking at the assets I'm replacing, it appears that I also have to make up an outpost officer as well, but she's only one person. I would love to make the game equal-opportunity, but the outpost officer takes up the space where the female heads would go. The female is actually a touch smaller then the male avatar.

Comparing the Commanding Officers

Anyways I will live you with a female mock up and I'll be doing the Admiral tomorrow. I've got a system down now so the characters I generate should be getting pushed out quicker and quicker. After the avatars are all done, then it's off to font editing and then planet rendering in blender for the spinney globe logo.

Look at me! I'm the Captain Now.

This is just a quick bonus before/after picture before I go to bed. As you can see, you can make a whole lot prettier pictures with four times the resolution. I 'm going to be starting on the female uniform tomorrow. I just thought I would show off the paper doll, with yours truly as the captain. I apologize as there really isn't much to espouse when I'm doing the art stuff. I guess the picture is worth a thousand words.  In the end I was lazy and decided to hide the hands.

Uniforms

I'm making a dent in the uniform for the male captain uniform. I have three major bodies to do. Male captain , Female captain , and the admiral who will be in a different pose then this one. The picture I have here is a work in progress, I still have to lay out the shadows, clean up the belt buckle and fix the lighting on the boots (It's backwards). I was never a fan of my ability to art. I seem to always be working by the skin of my teeth. The real hurdle is how much everything looks like a disaster until it's almost done. The head is going to be made with a bunch of swap-out generic parts swapped to give each face a distinct look.

I'm not sure about hiding the hands or not. I'm personally rubbish at them, however then is code that changes the hand color based on the race of the character you chose. Also I'm going to use a little creative license to fill out the shoulder patch. I'm most likely going to put a sun there.

After the officers are done, I'll be redoing the in-game font and replacing the gprint() command with Allegro's text. That becomes a bonus as I could use Unicode for non-western languages. (The ability would be there, but not implemented. Don't want to get too much feature creep). These bodies will make a return as later in the game you use them to create enemies as well.

Back to the Wacom I guess.

Programming on Accident

In my flurry of flu I attempted to make a little head way doing some much needed replacement of console and POSIX functions. The goal is to get rid of as many dependencies as I can and have Allegro as my core library for input and an output. As I wrote I was bent on getting rid of some very legacy console functions. Namely _kbhit() and _getch(). (Hello there underscore functions!)

It seemed simple. All _kbhit() did was return a 1 if there was a key in the keyboard queue, and a 0 of not. _getch() just grabs the next character in the queue. The problem I was having was I replaced the BIOS keyboard queue with Allegro's to get things functional. However, the new keyboard queue holds MUCH more than just what key was pressed. It also holds, in order, the key down and key up events. So just because something is waiting in the queue, doen't necessarily mean it's a character. I flailed with this for a while, making implementations of _kbhit() and _getch() that worked "good enough" while at the same time goofing up my keyboard queue. I got it to work in my test unit, and then when it was in the game, failed massively.

I was programming on accident.

For those who don't know, this is when you code without any idea what you are doing in hopes that it works. The problem is that if you do this, and the program breaks later, you have no idea what's wrong or how to fix it. It was also frustrating myself as I couldn't seem to get two fairly simple functions to work properly. I was pointed in the right direction with some help on a forum. Turns out I wasn't follow my logic properly. The cool thing was as long as I showed off my example code and made it obvious I was making an effort, people were bending over backwards to help and explain what I was doing wrong. Thumbs up!

The old paperdoll, layered in GIMP.

After I got my mojo back. I got rid of the master timer from time.h and moved that to Allegro's internal timer. I'm trying my best to do "drop in" replacements of the commands. Now I no longer need clock(), clock_t struct, or time.h  and have replaced those with my own functionality  as well.

Moving on I'm now working on the paper dolls that represent the commanders in the game. I'm fixing up the poses and character designs, but keeping the uniforms mostly the same. I'll do some initial character designs on my own, but may need backup if it's a little too time consuming.

Down with the Sickness

Oooh-Wa-ah-ah-ah~

I think I have the the flu that's going around. I wasn't able to get mobile till 3:00 this afternoon. I tried to salvage the rest of the day by doing a little side work. I'm still removing the legacy console functions and decided on replacing _kbhit() and _getch(). Turns out the Allegro queues do not work the way that these commands assume and are missing keystrokes are getting getting events stuck in the queue. When these are implemented the way I would like, I will move on to the more fun drawing and graphical updates. 

Screens

This was the only way I could think of to show off full screen

I spent most of the day getting the game to run in full screen. I had to do a few experiments with stretching, but in the end I decided to just run the game in the center and play shenanigans with the border. I tried to stretch the same over the whole display but it looked awful. I think 1280x800 is good and I'll doll up the border later. The big thing I fought with was the faders. I had to make my hacks a little more robust so they would work in windowed and full screen modes

The other thing I did was work on the audio so now I have a proper mixer for the music. The sound effects will be rolled in later when I start implementing them in the game proper.

Not much of an update. The fullscreen problem  seems like such a little thing, but it required a lot of refracting to get right. I'm happy with it now. There are just a bunch a little things to pick up now. Looking ahead, it appears I may have to start drawing some fleet commanders in HD. Let's hope I can make that work out. 

Productivity!

So I decided to tackle the audio system and have that semi-implemented now. I ran into a few snags that aren't really show stoppers, but I had to take a side track this morning. In the game there are 20 sound effects that are called. Depending on the hardware you had back then, a sample was played. If you had a Sound-blaster, a digital VOC file was played. If you had Adlib, which didn't have a DAC to play samples, it fell back to hand-manipulated sound effects. If you just had the old squeaker speaker, a sample was shoved out via the timer.

The problem was all the digital samples were not convertible into something more contemporary. The .snd file for the squeaker speaker appeared to be raw PWM data. The Adlib used assembly to hand-make the sound effects, and the Sound-blaster VOC files have their own special problems. It seems that even though back in the day VOCs were ubiquitous, they were also proprietary. Creative labs never released the actual format. There are a few programs that can read VOC files, but the ones I have are packed and not readable, or covertable by any of my legacy software.

This is not a especially terrible thing. The samples are rips from the original 60's Star Trek and need to be replaced anyway. I know where each effect is played in the game so I can hunt some sounds that are more appropriate for the game.

The last pass I have is allowing me to fix up Nermal once and foe all, and support high resolutions. One of my core functions, the blitter, is super hacky and I now have an opportunity to fix that, and most of the other functions, once and for all.


The credits roll in glorious 1280x800 now :)

Filesystem Abstraction

It's cold up here...

So there is a new weather satellite that was recently launched by NOAA that is beyond anything we currently have up there. This one is in geosynchronous orbit and has a high-resolution infrared camera so you can see weather patterns at night. During few days of cold I took a look at it and realized why it was dipping down past freezing for the last week or so. I wish Canada would find a better way of invading us.

Bright and early I was back into the code. Now that I have a good idea of what is what. I'm not as terrified to move things around and change up the program outside of Nermal. I have a few major goals this time around. The first I want to reduce the number of dependencies. First I'm abstracting the filesystem through Allegro and making it as generic as possible. The back-end, which currently points to my hard drive can be swapped out for any other kind of file system. This will make porting the game to Android/iOS a lot easier as the filesystems in those environments are abstracted and sandboxed anyway. In the meantime Not only do I find myself able to get rid of old DOS/Windows functions, but some POSIX ones as well.

I'm also getting rid of the insecure string functions and replacing them with their secure _s counterparts. The difference between srtcpy() and strcpy_s() is that the secure ones requires me tell it how big of a string buffer is I'm using. This is to stop buffer overruns from happening. The good news is all the strings I'm running into so far are not dynamically created so I can just pass the new functions the length of the buffers directly.

The audio functions are proving to be a bit more complicated than I thought. I also had to re-render the opening theme to Ogg Vorbis. Looks like mp3 support was not implemented due to patents. It's only very recently that those have expired. Doesn't really matter, they all sound the same.

I'm hoping tomorrow I'll have the opening done in 1280x800 with audio as a proof of concept. That should motivate me to start updating all the graphical assets and fix the 1Ch thread once an for all.

(I made an edit to this blog after the fact. My secure code looked awful!)

Music Man

Yea, I'm working on my day off, but the music is done. On my first attempt to convert the MT-32 MIDI into GM, I accidentally scrubbed the program change commands that made each channel change to a different instrument (Not every channel has just one instrument patch). Now that I had an event editor I manually changed each program change event to select the correct instrument. I then ran into another issue. I couldn't find a single SonfFont that sounded good with all the tracks. Either the orchestra hit was too loud, or just used the strings, or another had the harp slightly out of tune. In the end I selected which SoundFont sounded best for each track and exported all the tracks individually as a wave file. Then I imported all the tracks into Audacity and balanced each track to make it sound the best. After that, one big export into MP3 and the music conversion is complete. On Monday I'm be back to coding, starting with redefining the config file to allow for different resolutions, mouse settings, and volume controls.

Here's a snippet of the final music.

Competing Standards

Left General Midi, Right Roland MT32

Remember when I said that I couldn't recall if it was Roland or Yamaha who had the non-standard MIDI mapping. Turns out it was both. Call me old fashioned, but when I learned MIDI back in the day, it was with a Korg X1 and an Apple II running my sequencer. (Okay, I'm dating myself now). I just used straight, standard General MIDI. I never had to deal with the strange MIDI dialects from the 90s.  The original ROE title was composed with a Roland MT-32 in mind, and that does not adhere to the General MIDI for the most part. That's why it sounded so awful when I tried to play the title music "out of the box". There are all kind of strange weirdness for that MIDI device. First you can only have 9 tracks, not 16. These are numbered 1-8 and then R for the rhythm. These map to general MIDI track 2-9 and 10 for percussion. Also, the instrument patches are not the same. I created a spreadsheet to see to see what maps to what.

Even though Yamaha really isn't in this picture, that company's changes were a little more standard and some of the extensions made sense. For example, you could have more that 128 sounds, but made it so the percussion track wasn't hard-coded to the 10th track. You had to map a drum kit instrument to the track instead.

I contemplated just rendering the music though an MT-32 emulator. Sadly, the sound samples are copyrighted by Roland and they have been known to get litigious in the past about using their samples in an emulator. It appears I will need to slightly resequence the title music to have it conform to General Midi. This includes patching the instruments back, which I have done, but I also need to remap the controllers (Volume, Pan, Sustain) and scrub the proprietary Roland SysEx (System Executive) events. This is why my volumes, sustains, and pans were all over the place.

The good news is I have a better sequence editor. The other one I had was pretty basic and just gave me the ability to edit the piano scroll and some tracking elements. The one I have now has a track editor, piano scroll editor, notation, and the ability to manually edit the event timeline. It's the event editor that is the most useful. It's also the oldest of old-school ways of editing MIDI. Back in the day, the event list was all I had on the Apple II.

I'll be playing catch up the weekend and hopefully have a nice sounding title theme when I'm done.

Wearing hats

Getting Over It

What makes this whole ROE port interesting is that there is not a shortage of things to do. One of my bad habits is that I can tend to obsess about something and then do that thing until I burn out. After I was able to get the original ROE code to compile and limp along, the holidays hit. After that, I felt almost like the guy in the pot from "Getting Over It with Bennett Foddy". I'm going to have to crawl back up the mountain, this time in high resolution. Along the way pick up the detritus of code I've left behind just to get the system to function. This time it'll be in glorious 1280x800. As of right now, I'm not too keen on going back into Visual Studio, and following my call maps from Doxegen just yet.

This game does have me wearing different hats all the time. I can code, and was a database programmer one upon a time many moons ago. I'm rusty and flail sometimes, but can get things to work with effort. I'm not sure of that's how programmers are, but I've been told that is most often the norm. I have met with programmers who literally plug in headphones and zone for 16 straight hours. I've been in that zen in the past once or twice, but my brain doesn't make a habit of it.

I can also draw, and in my inventory have both a Wacom tablet and scanner. I'm okay  with GIMP and can do some of the more advanced mechanical stuff. That being said, never used Photoshop. I'm also pretty average in Inkscape, and once again, never used Adobe Illustrator. I can model 3D models, animate, and do video editing in Blender, but never touched 3D Studio Max. It seems I'm just one step short of "Pro", but It's a comfort zone I guess.

Arrows show the wrong patching

So today I decided to wear yet another hat and fix ROE's music.  The first thing I did was convert the MIDI file into sheet music so I could see what I was looking at. That way I could break down the movements and figure out what was channeled to what. The sequencer I have is a bit clunky and I wasn't getting the info I needed by looking at the piano scroll.

On import, I immediately found out what the problem was. It was channeled all wrong. I have a screenshot here showing that what each track was being labeled as, and what channel it was using was way off. I'm not sure why it was like this, but I fixed it up.

A little MIDI 101:

MIDI files are basically digital piano scrolls. They simply tell an instrument when and how to hit a note and for how long. They don't have any audio in them though. Instruments can be physical musical keyboards, drum machines, or digital synthesizers. Nowadays MIDI is also used for controlling stage lights and things like that due to their event-like system. MIDI has "tracks" that you put notes on and maps to one of 16 "channels" where you can, by convention, select one of 128 different instruments or sounds. The issue with ROE's title music was the tracks were not mapped to the proper instrument. (You can also change instruments on the fly in a track using an event, but that's me getting ahead of myself)

The default playback instrument is what you have plugged into the MIDI system. Back in the day, the Adlib and Soundblaster sound cards emulated the 128 interments as best they could with the OPL2 chips they had. Nowadays, Windows (Well, DirectX to be more specific ) comes with a digital  wave table synthesizer by Microsoft that uses digital samples for instruments. It sounds... well, fairly bland. You can "plug in" your own synthesizer, which is what I've done, and use SoundFonts to get better samples.

Soundfonts are kind of a lost art now. In fact, many of them are found online packed in a format that you decompress with a program that's not made anymore. By luck I found it, but wow. Talk about locking your keys in the car!

Right now I'm trying to find the best SoundFont to render the music with. I may have to make my own as each sound a little strange for a particular channel. After that I will finally dive back into Visual Studio and re-do the title sequence in high resolution with proper music and everything.

Here is a sample of before (OPL2) and after (Wavetable) when it comes to the title music It's not done yet, but sounds much better :)

Low Key

My work ethic was in the toilet today. Getting back into working after the holiday is kind of a drag. I finished up the HD graphic assets for the title sequence, and poked around at the MIDI opening music. It's constructed really weird. It seems that the better the quality SoundFonts I throw at it, the worse it sounds. I know it sounds "fine" with the Yamahama YM3812 synth. (As fine as it can get anyway). The Game was designed for Roland MT32, so I may have to go down the MIDI rabbit hole and see if the tracks are are not channeled correctly. I can't remember if it's Yamaha or Roland that "broke" the General Midi rules about percussion channel placement. It's supposed to be channel 10, but one of those two did an extension and moved the drums around. I mention this because while playing the opening, I'm hearing what is supposed to be timpani drums are coming out as tubular bells. There is also the most obnoxious "whoosh" sound coming from the score at random times. I'm wondering if that's not supposed to be the credits coming in and out, or did a track get stuck on "applause". I'll peek into this tomorrow.

On the flip side I figured out how to override the Microsoft GS Wavetable Synth with a software synthesizer that supports SondFonts. This means I can edit the MIDI sequence and play it back in my editor with the correct sounding instruments . After I'm done I'll be able to render it to MP3 with Fluidsynth and the same SoundFont.

Looks better then the CGA version

Powers of 10

In 1977 there was a popular science film from IBM called "Powers of 10". It started with a view of 1 meter square and scaled up to the observable universe. Then the film took us back down to 1 meter, and started zooming into the microscopic world until we arrived inside a proton. You can click the link to go to the video on YouTube. What's interesting is how much science had progressed since the film was made. To give an idea of how far we have come, the move signposted 10²⁴ meters (100 million light years) as the edge of the observable universe. Nowadays we can see to the Cosmic Microwave Background radiation, which was mapped with NASA's COBE satellite in 1989. The CMB is about 46 billion light years away which is 4.4x10²⁶ meters. As the universe is expanding, the furthest we will ever be able to see is 65 Billion light years, which is 5.9x10²⁶. (We actually can't see that far because the CMB is in the way, and the universe was opaque before that time). As the universe expands, that number is going down. (Every day we see less and less of the observable universe because it's expanding faster than light)

In the other direction, the video goes to 10^-15 meters. The smallest detectable thing we have now is the width of a 1 MeV neutrino which is 2 × 10^-23 meters. This was detected with the Large Hadron Collider. However, the smallest thing that can be is plank length, which is 1.62×10^-35 meters. This is currently being measured with the help of a watt balance to define a kilogram.

The point of this whole story is that I'm now working on the high resolution assets for ROE. One of the things I did was correct a few dates. Science progressed a bit since 1991. In a previous post I mentioned the opening titles said that the universe was 16 billion years old. It's actually closer to 13.82 billion. The credits also said that man emerged 4 million years ago. Last year a scientific paper came out that showed humans split from the Least Common Ancestor of chimps 12 million years ago. The split took a particularly long time to complete because there was a lot of genetic transfer during that time. Anyways, I corrected that date too.

I'm pretty happy with how the updated graphics are looking. I do have some plans for the humans and aliens in the game and how they are going to look. I'm trying to keep the look and feel as close to the original game as I can, but those human designs are going to get a much needed overhaul. (While keeping the same uniforms)

The opening music will be an MP3 rendering of the original MIDI. Digital version I have right now was played on an emulated Yamaha YM3812 chip that comes on a SoundBlaster. I tried to re-render the MIDI it using FluidSynth and an orchestra Soundfont. The output was pretty much a disaster, but the parts that were good were awesome. I'm probably going to re-sequence the original MIDI so it will render better. The events are fine, but I'll need to re-channel the tracks to better match the patches in the Soundfont.

Happy New Year!