## Wednesday, 24 February 2010

### Plotting in 6 dimensions - parametric plot from a file

Today I would like to touch on a vast subject, so prepare for a long post. However, I hope that the post will be worthwhile, for I want to discuss something that cannot be done in any other way. In due course, we will see how we can use gnuplot to create parametric plots from a file. What I mean by that is the following: if you want to plot, say, 10 similar objects, whose size is determined by the first column in a file. Of course, there are cases, when one can manipulate the size, e.g., if there is a pre-defined symbol, we can use one of the columns in a file to determine the size of the symbol. As an example, we can do this
```plot 'foo' u 1:2:3 with point pt 6 ps var
```
which will draw circles whose radius is given by the third column in 'foo'. This is all well, but it works for a limited number of cases only, namely, when there is a symbol to start out with. But what happens, if we want to draw an object that is not a symbol, e.g., arcs of a circle, whose angle is given by one of the columns in a file, or cylinders, whose height is a variable, read from a file. As you can guess from these two suggestions, what we will do is to draw a pie, and a bar chart. I understand that we have done this a couple of times before, but this time, we will stay entirely in the realm of gnuplot, and the scripts are really short. We just have to figure out what to write in the scripts. But beyond this, I will also show how we can plot in 6 dimensions. We will plot ellipses on a plane (first 2 columns), whose two axes are given by the 3rd and 4th axis, the orientation by the 5th, and the colour by the 6th. If you are really pressed for it, you can add three more dimensions: if you draw ellipsoids in 3D, take all three axes from a file, and also the orientation, that would make 9 dimensions altogether. Quite a lot!

So, let us get down to business! The first thing that I would like to discuss is the evaluate command. This is a really nifty way of shortening repetitive commands. Let us suppose that we want to place 10 arrows on our graph, and only the first coordinate of the arrows changes, otherwise everything is the same. Setting one arrow would read as follows
```set arrow from 0, 0 to 1, 1
```
Of course, there are quite a few settings that we could specify, but this was supposed to be a minimal example. Then, the next arrow should be
```set arrow from 1, 0 to 2, 1
```
and so on. What if we do not want to write this line a thousand times, and we do not want to search for the coordinate that we are to change, the first one, in this case? We could try the following
```a(x) = sprintf("set arrow from %d, 0 to %d, 1", x, x+1)
```
This function takes 'x', and returns a string with all the settings and coordinates. So, we are almost done. The only thing we should do is to make gnuplot understand that what we want it to treat a(x) as a command, not as a string. Enter the eval command: it takes whatever string is presented to it, and turns it into a command. Thus, the following script creates 5 arrows, all parallel to each other, and consecutively shifted to the rigth
```a(x) = sprintf("set arrow from %d, 0 to %d, 1", x, x+1)
eval a(0)
eval a(1)
eval a(2)
eval a(3)
eval a(4)
```
I believe, this is a much simpler and cleaner procedure, than this
```set arrow from 0, 0 to 1, 1
set arrow from 1, 0 to 2, 1
set arrow from 2, 0 to 3, 1
set arrow from 3, 0 to 4, 1
set arrow from 4, 0 to 5, 1
```
I should mention here that if chunks of a command are the same, another method of abbreviating them is to use macros. Those are disabled by default, so first we have to set it. Then it works as follows
```set macro
ST = "using 1:2 with lines lt 3 lw 3"
plot 'foo' @ST, 'bar' @ST
```
i.e., the term @ST is expanded using the definition above, therefore, this plot is equivalent to this one
```plot 'foo' using 1:2 with lines lt 3 lw 3, 'bar' using 1:2 with lines lt 3 lw 3
```
but the previous one is much more readable. I would also say that using capitals for the macros is probably not a bad idea, because then they cannot be mistaken for standard gnuplot commands. This much in the way of macros!

So, we have the evaluate command, and we have a new concept for functions. Then let us take a closer look at the following code
```a(x) = sprintf("set arrow from %d, 0 to %d, 1;\n", x, x+1)
ARROW = ""
f(x) = (ARROW = ARROW.a(x), x)
plot 'foo' using 1:(f(\$1))
```
and let us suppose that our file 'foo' contains the following 5 lines
```1
3
5
7
9
```
After plotting 'foo', the string 'ARROW' will be the following
```set arrow from 1, 0 to 2, 1;
set arrow from 3, 0 to 4, 1;
set arrow from 5, 0 to 6, 1;
set arrow from 7, 0 to 8, 1;
set arrow from 9, 0 to 10, 1;
```
I.e., we have a string, which contains instructions for setting 5 arrow. If, at this point, we simply evaluate this string, all 5 arrows will be set. Therefore, we have found a way of using a file to set the coordinates of an arrow. (N.B., if it was for the arrows only, we wouldn't have had to do anything, since there is a plotting style, 'with vector', as we discussed some weeks ago.)

We will use this trick to create a parametric plot, taking parameter values from a file, first plotting the ellipses! Again, we have got to create some dummy data, and since we now need 6 columns, we will use the errorbars

```reset
f(x) = rand(0)
set sample 50
set table 'ellipse.dat'
plot [0:10] '+' using (20*f(\$1)):(20*f(\$1)):(f(\$1)):(f(\$1)):(3.14*f(\$1)):(f(\$1)) w xyerror
unset table
```
which will produce 6 columns and 50 lines. Having produced some data, let us see what we can do with it. Here is our script:

```PRINT(x, y, a, b, alpha, colour) = \
sprintf("%f+v*(%f*cos(u)*cos(%f)-%f*sin(u)*sin(%f)),
%f+v*(%f*cos(u)*sin(%f)+%f*sin(u)*cos(%f)),
%f with pm3d", x, a, alpha, b, alpha, y, b, alpha, b, alpha, colour)
PLOT = "splot "
num = -1
count(x) = (num = num+1, 1)
g(x) = (PLOT = PLOT.PRINT(\$1, \$2, \$3, \$4, \$5, \$6), \
(\$0 < num ? PLOT=PLOT.sprintf(",\n") : 1/0))
plot 'ellipse.dat' u 1:(count(\$1))
plot 'ellipse.dat' using 1:(g(\$1))

unset key
set parametric
set urange [0:2*pi]
set vrange [0:1]
set pm3d map
set size 0.5, 1
eval(PLOT)
```

First, we have the definition of a print function that looks rather ugly, but is quite simple. We want to plot
```a*v*cos(u), b*v*sin(u), colour
```
where a, and b are the axes of the ellipse, and colour is going to specify, well, its colour. However, we want to translate the ellipse to its proper position, and we also want to rotate it by an amount given by the 5th column, so we have to apply a two-dimensional rotation on the object. Therefore, we would end up with a function similar to this
```x+v*(a*cos(u)*cos(alpha)-b*sin(u)*sin(alpha)), y + v*(a*cos(u)*sin(alpha)+b*sin(u)*cos(alpha)), colour
```
Now you know why that print function looked so complicated! After this, we define a string, PLOT, that we will expand as we read the file. But before that, we have to count the lines in the file. The reason for that is that successive plots must be separated by a comma, but there shouldn't be a comma after the last plot. So, we just have to know where to stop placing commas in our string. Then we define the function that does nothing useful, but concatenates the PLOT string as it reads the file. Here we use the number of lines that we determine in a dummy plot. At this point we are done with the functions, all we have to do is plotting.
First we count, then plot g(x). At this point, we have the string that we need. We only have to set up our plot. Remember, we have a parametric plot, where the range of one of the variables is in [0:2*pi], while the other one is in [0:1]. Easy. Then we just have to evaluate our plot string, and we are done. Look what we have made here: a six-dimensional plot!

I think that this script is much less complicated, than many that we have discussed in the past. Short and clear, thanks to the eval command, and the new concept of functions. Besides, we pulled off a trick that was impossible by other means. I started out saying that we will create bars and pie. I believe, having seen the trick, it should be quite simple now, but in case you insist on seeing it, I will discuss it in my next post.

1. Nice.

However, it is not entirely true that there is no other way to produce a plot like this.

Please take a look at my patch at Sourceforge:

http://sourceforge.net/tracker/?func=detail&aid=2953926&group_id=2055&atid=302055

The patch adds a new 2D plot style "with ellipses" that makes such plots much easier to produce.

In fact, a simple
plot 'ellipse.dat' u 1:2:3:4:(180/pi*\$5):6 with ellipses lc pal
would do it.

It's not yet in the CVS, but you can help persuade the developers to commit it :)

(In the patch you will find a demo that implements a fully rotatable 3D Solar System orbit viewer, given a file of orbital elements. It uses a similar trick of building the plot command from a data file then executing it, but I used perl to build the command.)

I also supplied an other patch earlier that extends the existing 'circles' style: you can now draw arcs with explicitly specified start and end angles, not only full circles. This feature is already committed to the CVS tree.

By the way, your posts on this blog are very useful, I've learned a lot from them. Keep up the good work!

Péter Juhász

2. First, thanks for the kind words! Second, I have just written a reply to your post on the discussion group, without reading this, so that is sort of obsolete now. (This whole idea occurred to me when I saw your first comment a couple of weeks ago, and then someone mentioned that they wanted to plot galaxies, or what not.)
But I would still maintain that while the 'with ellipse' flag is certainly useful, it is limited to ellipses only. Also, it cannot be expanded, while this method can. (I can easily add two more dimensions, although I don't see the pressing need for it.)
My problem with patches is that the development cycle appears to be a bit long (I am not blaming anyone here, mind you!), so whatever you implement now will make it to the main branch in 3 years' time. That is a bit too long. However, I would love to see an implementation of parametric plot from a file, in a sense as I outlined in this post, so if you feel like collaborating on that, I would be interested.
Cheers,
Zoltán

3. I agree that this method is much more general and customizable than the one made available by my patch. However, it is also fairly complicated - this is not something you'd want to show a beginner.

And the very point of my patch was that the functionality (ellipse drawing) was already present, so by creating a new interface to it a relatively useful capability could be added to the program with relatively little effort.

I also agree that the development cycle may be too long. But this is open source after all: you are free to get the development version and modify it for your personal use as you need. Sending a patch in and thus enabling others to profit from your work is optional.

By the way, why are we writing in English? :)

Péter Juhász

4. Gnuplotter,

Thanks for your wonderful gnuplot examples. Your "Plotting in 6 dimensions - parametric plot from a file" is of particular interest to me, as I want to do something very similar.

Instead of ellipses, I want simple rectangles that I can control the color of. I also want to specify the rectangle top, bottom, left, right (or similar). All the rectangles will be parallel to the plot axes. So the ellipse example is a great starting point for me.

Unfortunately for me, I'm new to gnuplot. I've been studying the documentation, but I cannot find various features which seem to be crucial to your example.

Here are some newbie questions for you –

1) The part of the script that creates the ellipse.dat file has a ‘+’ as the file name. What is the ‘+’ (in single quotes) doing? I cannot find anything like this in the documentation.

2) The “using (20*f(\$1)):(20*f(\$1)): etc.” stuff is a mystery to me also. I’ve looked at the ellipse.dat file, but I cannot figure out the relationships between the functions and their corresponding columns, except that there are 6 columns of numbers.

3) It would be most convenient for me if the plot data and the plot commands were all in the same file. I will be generating the file using other software; conceptually, the result will be the chart, not the intermediate file(s). Is there a way to combine the data and plot commands into one file? I’ve seen some hints that this may be possible, but nothing that actually shows how to do it.

4) Is it possible to produce a chart that has scroll bars? I’d like to produce a very tall graph that can be scrolled, without having to replot.

Thanks so much for providing these detailed examples.

Regards,
Mark.

5. Greetings Mark,

Thanks for visiting! You had quite a few questions, I will try to answer them.

> 1) The part of the script that creates the ellipse.dat file has a ‘+’ as the file name. What is the ‘+’ (in single quotes) doing? I cannot find anything like this in the documentation.

The '+' is a pseudo-file, and the purpose of that is that by using this, one can use plot modifiers for functions. E.g., if your colouring of a surface depends on the difference of the first and second column, you could not do this with a function, because a function plot looks like

splot f(x,y) with pm3d

or something similar. If you had a file containing three columns, then you could plot as

splot 'foo' u 1:2:3 with pm3d

Now, if you want to colour your surface as a function of \$1-\$2, you can do this

splot 'foo' u 1:2:3:(\$1-\$2) with palette.

Since the introduction of the '+' pseudo-file, you can do the same thing with functions, as

splot '+' u 1:2:(f(\$1,\$2)):(\$1-\$2) with palette

You can find the documentation of this in gnuplot-4.4.rc1.pdf

> 2) The “using (20*f(\$1)):(20*f(\$1)): etc.” stuff is a mystery to me also. I’ve looked at the ellipse.dat file, but I cannot figure out the relationships between the functions and their corresponding columns, except that there are 6 columns of numbers.

I think, my comment above answers this question.

> 3) It would be most convenient for me if the plot data and the plot commands were all in the same file. I will be generating the file using other software; conceptually, the result will be the chart, not the intermediate file(s). Is there a way to combine the data and plot commands into one file? I’ve seen some hints that this may be possible, but nothing that actually shows how to do it.

This I wouldn't do. You can generate your data, foo.dat, and have a script file, foo.gnu, foo.gp, foo.plot, whatever, that plots your data file. However, if you insist, you can do what you want. All you have to do is to use the '-' pseudo-file. This does nothing but reads the command line. So, your file could be something like this

plot '-' using 1:2 with lines
1 10
2 5
3 12
4 7
5 3
e

This plots the data as usual. Just don't forget to close your data set with a trailing 'e' at the end, otherwise your gnuplot session will keep waiting for input.

> 4) Is it possible to produce a chart that has scroll bars? I’d like to produce a very tall graph that can be scrolled, without having to replot.

You can try to play with the canvas terminal, which was introduced in gnuplot 4.4.
Cheers,
Zoltán

6. Thanks so much for your kind reply, and thanks for putting together this wonderful blog.

With your help, I was able to find the "+", etc. stuff in the documentation. This is in section "65.2.7 Special-filenames" of gnuplot-4.4.0.pdf, for others who may need the info.

Your other answers were also all very helpful; I think I now have enough information to try some things. It may be a while before I get to since some other things currently have higher priority.

Thanks again.
Mark.

7. Recently use your exotic fashion loop to plot a line progersively in separeted files.
These allow to animate the plot and syncronize with a movie. That's the code:

a(x) = sprintf("\
set output name.'%d.png';\
plot '< head -%d NebF.NEB2.dat' u 1:4 with line;\
unset out ; \
",x,x)

LOOP = ""
f(x) = (LOOP = LOOP.a(x), x)
plot '< seq 1 100' u 1:(f(\$1))

eval LOOP