What I proposed was an editable text file of information. Not true binary, but representative text.

true binary in the 8 bit avenue has 256 bits available. No where did I propose using all 256 outcomes.

It means the simplicity of the text file would make it so easy to make a wide variety of possible map gens from a wide variety of angles. I could make a map using excel and MSWord for instance, with ease, from such a text file. Simplicity is important in an application.

If you want something complex, with maximum size savings, just say so.

> And with a format as proposed by flint, it would still be readable for
> the users. Even better readable than an xml file, because you could get
> a visual representation of the map "in ASCII art". And the file size would
> still be very small.

No, lol. This would be giving up on speed and space, as well as giving up on
the very readable and editable XML. Basically this "ASCII art" would have the
negatives of both binary format and XML format.

As for the "visual representation" part -- thats the point of the map software.

> I dont quite get why a binary file or a two dimensional text file would be bad.

Its not.

XML is just more "standard-friendly". Its much easier for anyone with even a
little bit of experience to open an XML file and say "Hey, this does that". If you
do the same with a binary file you either need to be prepared (documentation) or
spend time trying to decrypt exactly what each piece of data does.

skoe:

"Well, thats true. But what if game X implements something ever so slightly different -- and we both create clients that use this format. Your client might work fine, but mine doesn't know about the extra features of game X."
You're talking about unofficial extensions to the XML (eg: families can claim secondary homesystems (an actual core), or a 1 on 1 galaxy without families is being played). Assuming the minimum is implemented to begin with, any extra information contained in the file is at worst ignored. That's forward compatibility.

One reason I'm hesitant to add a game_id tag is the potential for name collisions/duplication. Is this game IC or ic or or "Imperial Conflict"? What do we do with Ectroverse, where anybody can run their server? Everyone can't have the same tag, but how do we stop duplication? And is the mapgen program really going to have to understand all of these different possible tag/attributes combinations on a per-game basis?

Do you think it makes sense to allow extending the file the same way you can for sitemaps?
http://www.sitemaps.org/protocol.php#extending
Then a particular program could decide from the new xmlns whether or not it supports the extra stuff. I'm not very familiar with XML namespaces, so I don't know exactly how that would work.

"Another thing you could include: The author of the map file (who created it). Its not really that important in the grand scheme of things, but its still nice to give credit where credit is due. For example i think i read somewhere that you (Smartys) created a mapfile for Capri. Credit should be given to you within that file."
Yes, it is. However, like I said, I envision a future where creating this file is done entirely automatically or is made to be much less of a chore than it is now. The author is really non-important in that context.

---

Flint:

"It means the simplicity of the text file would make it so easy to make a wide variety of possible map gens from a wide variety of angles. I could make a map using excel and MSWord for instance, with ease, from such a text file. Simplicity is important in an application."
And what happens when I decide to add a new attribute to systems, to mark them as uninhabitable (one possible game-specific extension). How would your format cope with that? I'm still not sure how your format copes with homesystems, or how it stores other galaxy meta-data (eg: number of systems, number of sectors).

XML is never going to win on the basis of size or parsing speed. However, where it excels is readability and extensibility. Your format would definitely result in a smaller file, especially compressed. Also, as skoe pointed out, XML parsing requires parsing the entire document first and then inferring meaning, whereas with a binary file format you always know where you are (unless someone followed the spec wrong ). However, parsing has to be done once, when the file is opened. It doesn't need to be done 100x per second. And, unless I'm mistaken, I can't just add random attributes to systems in your format and still have my file work.

Also, saying that you don't need documentation to understand your file is disingenuous. It may seem that way to you because you came up with the format, but someone opening up the file would have very little idea what the format is without documentation of some kind. For instance: who's to say the 0s aren't systems, since 0 is associated with success for executables in Linux? At best, trial and error would be needed if there were documentation (it took me a lot of work to get the current map file format right, even though I had a well documented generator to base my code off of). XML, on the other hand, is largely self-documenting. There will be an xmlns file at some point which provides explicit documentation, but the tag names themselves carry a lot of meaning. People can understand what editing certain attributes, removing certain tags, etc does.

---

Arachnophobia

"I dont quite get why a binary file or a two dimensional text file would be bad. The users arent supposed to read it anyways, just the program, which can be independend from systems. And with a format as proposed by flint, it would still be readable for the users. Even better readable than an xml file, because you could get a visual representation of the map "in ASCII art". And the file size would still be very small."
Extensibility is, I would say, the biggest advantage of XML. If a new feature is added to a game (I've mentioned a few examples earlier in the post), new tags or attributes can easily be added to deal with them. Not so with a binary format.
Size isn't really a factor, since the worst case XML is unrealistic and still compresses very nicely.

Lets compare the two formats:
- Size: XML is larger than the binary format, even when both are compressed. However, we're not talking about GB: we're talking about < 100 KB in most cases.
- Readability: XML is largely self-documenting, your format can be understood if some minimal documentation is provided.
- Parsing speed: XML is slower to parse than the data in a corresponding binary format. However, while the magnitude of the difference is unknown, my assumption is that it wouldn't be long enough to dissuade anyone from using the XML. Also, a mapfile is only parsed when it's loaded: all this does is slow down the application start time a little bit.
- Extensibility: XML wins, hands down. And this, I would say, is one of the more important features of the mapfile

Am I missing something?

How important is extensibility? Honestly.

Ease of understanding would have allowed me to create the current ICPG map myself if it was made in a format as I propose.

My concept would be:

The file would start with the galaxy name, width, length, sectors. Now the map begins and you can number all of the bits (or the program does to calculate the family positions). Every bit can be either 0 - no system, or 1 - yes system. After that, you can put family numbers with the corresponding number for placement when you count through the bits. That way you get to represent the whole map plus family positions.

Extensability
If there ever will be more properties for a system than it is existing (1 in the bit) or it being a home system (in the family table), you can just add more tables to the file, right where the family placement was noted.

Size
Should be pretty small no?

Accessability
Shouldnt be hard to understand. And all the "creep data bits" are generated by the computer anyways.

In any case, even though I can't write out the file I think I understand what you're going for. What you're describing is very similar to the current format used. In fact, it's almost identical.

I don't think the format is as easy to understand as you assume it is. You would definitely need some kind of external reference spec to tell you what each "table" means, as well as what the values mean.

There's also a flaw with your extensibility argument. If I add another "table", the program has to know what the next "table" it's reading is. With ICMapGen, the last element of the file is the vertical sector count, which wasn't included in the original format but was tagged on so it could be used in later versions of the program. If you allow arbitrary extensibility, you need some way (an ordered header of identifiers perhaps?) to indicate what the tables are so the program can derive some meaning from them. I'm going to assume the issue of skipping unknown data is moot, since you can easily include a special character to indicate the end of a "table".

Having to add a whole new table is also a lot less flexible than just adding an attribute to some systems or a new element or two.

No. But the average user is not going to be doing that, they're going to be using tools that use the data. And the ones that will want to manipulate the data will find that they can do much more with it as an XML document.

Not a binary file. I will pay you $100000000 dollars if it is actually a binary file. I proposed a text file with plain text characters. Anyone who read a readme could easily understand it.

Now apologize for calling it undecipherable and all that X(

> One reason I'm hesitant to add a game_id tag is the potential for name collisions/duplication. Is
> this game IC or ic or or "Imperial Conflict"? What do we do with Ectroverse, where anybody can
> run their server? Everyone can't have the same tag, but how do we stop duplication?

See the second part. You don't support "websites", you support "games".

> And is the mapgen program really going to have to understand all of these different possible
> tag/attributes combinations on a per-game basis?

No, you understand a particular games special extensions. If you do not understand the specific
extensions you alert the user "OY YOU, DONT SUPPORT, K?". The user can then exit your program
or accept that certain features are unavailable and proceed anyway.

In terms of "games":
For example IC might be open-sourced one day. If both of us have IC, and make modifications,
then we can't really call it IC anymore can we? But if only one of us makes modifications, then the
one who hasn't can still call his game "IC".

Also this standard needs a way of identifying itself. E.g Is this standard 1.1 or 1.2? Or maybe we use
an incrementing character? (A, B, C).

OK, so given a slightly enhanced understanding of XML namespaces, here's how I would tackle this problem of multiple featuresets:

Game A has a namespace URI of http://example.com/a/1.1/
Game B has a namespace URI of http://example.com/b/1.2/
Game C has a namespace URI of http://example.com/c/1.0/
There is some default namespace URI, http://example.com/1.6/
(keep in mind that these are made up URIs).

A program has a static list of what namespace URIs it understands. If it encounters an element with a namespace that it does not understand, it prompts the user. Otherwise, it handles that element based on its understanding of the namespace. This allows games to mix/match features (or, to have separate namespaces for each feature). These namespaces are reusable by other games, so if I develop an IC-like game I can use IC's namespace URI (and include additional namespaces for new features if I so desire).

Note: namespaces have nothing to do with validation. They essentially just give a prefix to the element names so they don't conflict with one another.

Anyway, the advantage of this over a game_id element is that namespaces are reusable and game IDs are not. Instead of having a different set of parsing logic for each game, you have additional parsing logic for each namespace and call the parsing logic that you need.

Does that make sense to anyone other than me? Should I write up a demo XML file?