GeneralData

Description - This data source can be used to read in almost any form of text of binary data. When the user clicks to read a file, he/she is first presented with the option of what file to read. After this, a second dialog box comes up that asks for format information. This dialog has two primary options. If "Basic Text" is selected, the reader reads each line of the file as a new element and any numbers that are integers become parameters while any numbers that aren't in integer format become values. So for example, the line "5 7.5 8.4 2" would have p[0]=5, p[1]=2, v[0]=7.5, and v[1]=8.4.

For more power use the "Special Format" tab. To use this the user needs to supply a format string in the text area. The string is made from one of more lines each of which has one or more tokens on it. The tokens can be grouped by curly braces as well. Tokens come in the following flavors: stor=type, stor=#, type, or {#|stor, tokens}. In which stor can be one of three possible things. It can be p[#] for a parameter, v[#] for a value, or a variable name. A type begins with an i, r, or s and can be optionally followed by the size descriptors that are used for binary format files to give i*2, i*4, r*4, r*8, r*x4, or r*x8. The i is for an integer, r is for a real, and s is for a string. The number tells how many bytes it is if it is a binary file. Also, the x signifies that it uses the xdr format. A j can be placed after the * to signify that the reading should be done with Java binary reads. This is generally only correct if the file was written by Java. If only the type is provided, the value is read and thrown away (not stored in any part of the element). The use of variables is typically for integers only and it specifies how many times a subgrouping is supposed to be read. To allow reading multiple values or parameters of the same type the special syntax v[1..100]=type is also supported. This will read in 100 of the given type and put them into values 1 through 100.

If v[#] or p[#] is used with an =s to read in a string, that string is set to be the description of that column of values or parameters in the data set. This can be handy if you have a header line in the file. See below how to deal with reading only one header.

The variables and curly braces are used in conjunction to allow you to read in a number of elements that isn't known beforehand, but is stored in the file itself. It can also be used as shorthand when the number to be read in is known beforehand. An example would be n=i {n v[0..5]=r}. This will read in an integer that tells it how many elements to read where each element has 6 values in it. An element is returned every time the parsing gets to the closing of a maximally deep curly brace or the end of the string. Curly braces can be nested, though doing so represents 2-D data which this tool is currently not ideally suited for dealing with. Reading values between one close curly brace and a second close curly brace will produce unspecified results. Using a number after the open curly brace instead of a variable will always read that number of records. Parameters and values can also be used to specify the number of times a grouping should be read. If a value is used, it will be converted to an integer for that purpose. You can also put in a # to say that the given grouping should be read until the end of the file. This is a good way to skip header lines. You read in the strings in the header then follow that with a group that will be read to the end of the file. Variable names should only include characters and numbers.

Lastly, you can store numeric values directly into parameters or values by having a token of the format p[#]=# or v[#]=#. So if all elements read at a certain point should have a first parameter of 1 you might use p[0]=1.

Here is a more complex example of how something like the original SWIFT binary files might be specified in this format if written in real*4 format. This is similar to what happens in the BinaryPositionData class only it would require reading one fewer bodies which this class currently does not support.

v[0]=r*4 nbod=i*2 ntp=i*2
p[0]=0
{nbod p[1]=i*2 v[1..6]=r*4}
p[0]=1
{ntp p[1]=i*2 v[1..6]=r*4}

Note the use of stored constants to signify massive bodies or test particles. Also note that the time value read in at the very beginning is remembered all the way through until it is read again many elements later. This will be read repeatedly until the end of the file is reached. To read a smaller number, the entire string could be put in curly braces with a constant specified at the beginning. So this would try to read only 100 entries or stop at the end of the file if it was shorter than that. This is the one case here nesting curly braces is generally useful.

(100, v[0]=r*4 nbod=i*2 ntp=i*2 * p[0]=0
{nbod, p[1]=i*2 v[1..6]=r*4}
p[0]=1
{ntp, p[1]=i*2 v[1..6]=r*4}}

White spaces are used to separate tokens but are otherwise ignored. The format modifiers (*2, *4, etc.) are ignored for text input. For binary files the defaults are i*2 and r*4 as those are what are generally used by * SWIFT. Note that Fortran binary files typically have headers and footers around each record. Those are four byte ints that tell how many bytes are in the record. So to make the above actually work with a binary file produced by SWIFT (which you don't need to do since we provide that tool) you would need to add in extra reads of 4 byte integers as follows.

i*4 v[0]=r*4 nbod=i*2 ntp=i*2 i*4
p[0]=0
{nbod i*4 p[1]=i*2 v[1..6]=r*4 i*4}
p[0]=1
{ntp i*4 p[1]=i*2 v[1..6]=r*4 i*4}

To figure out where the extra integers are in the file, you have to see the code that was used to write it. Binary files written in C/C++ will not have this.

Elements - The format of the elements of this source vary depending on what options are specified.

Memory - This source does buffering of input that is bigger than what is specified in the options. For this reason, if you have very large files that you are reading, you should put a filter directly after this source that removes many of the elements. Selection and thinning filters are ideal for this.