Ribbon charts - without the gawk script

Yesterday, we saw how we can produce a ribbon chart from an arbitrary data file, and with the help of an external script. You can still make it work under windows, although you might have to call the script outside gnuplot. Probably, it would be better to do everything in gnuplot, wouldn't it? So, can we do something to alleviate the situation. But, of course, we can, we just have got to use those little grey cells! If you are patient and read further, I will explain how you can draw this graph completely automatically! (Uhm, this is rather similar to that from yesterday... Never mind, the point is that you haven't got to do any work!)

If you recall, the key to plotting the ribbons was to convert the original data file (see the beginning of my previous post) into something like this:

0 0 0.857354
1 0 0.857354

0 1 0.551386
1 1 0.551386

0 2 0.243002
1 2 0.243002

0 3 -0.24
1 3 -0.24

0 4 -0.422674
1 4 -0.422674
...

i.e., we need two rows of three columns, separated by a linefeed. We will need some data processing for this, but we can manage. First, we will duplicate all columns, simply plotting into a file. By issuing

set table 'rib1.dat'
plot 'ribbon.dat' u 1:1
unset table

we get the following file, 'rib1.dat'

#Curve 0 of 1, 14 points
#x y type
0.857354  0.857354  i
0.551386  0.551386  i
0.243002  0.243002  i
-0.24 -0.24  i
-0.422674 -0.422674  i
-0.513352 -0.513352  i
-0.660823 -0.660823  i
-0.0962106 -0.0962106  i
-0.0152513 -0.0152513  i
0.489453  0.489453  i
0.876023  0.876023  i
0.945061  0.945061  i
0.860648  0.860648  i
0.901743  0.901743  i

This is almost what we want, two columns of the same numbers, namely those from the first column of 'ribbon.dat'. The only difficulty is that it contains a third column, which in this case is just a set of i, but if we were to plot this file as a matrix, at x=2 all values would be equal to zero, so it would be a distorted ribbon. How can we get rid of that letter? Well, we will turn the into a form similar to that at the beginning of this post, simply by

set table 'rib2.dat'
splot 'rib.dat' mat
unset table

where now 'rib2.dat' looks like this

#Surface 0 of 1 surfaces

#IsoCurve 0, 3 points
#x y z type
0  13  0.901743 i
1  13  0.901743 i
2  13  0 i

#IsoCurve 1, 3 points
#x y z type
0  12  0.860648 i
1  12  0.860648 i
2  12  0 i
...

which is very similar to the desired format, for now we can simply call an splot, without the matrix specifier. But we still have an erroneous value in every third line. We will remove that using the 'every' specifier of plot. If you haven't done it yet, you should definitely issue the command

?every

You will learn a lot of interesting things. What we want to do here is to plot only the first end second record in each block. In that case, it does not matter what is in the third record, because it will not be processed by gnuplot. So, we take our 'rib2.dat' file, and plot it as

splot 'rib2.dat' every ::0::1 u 1:2:3 w pm3d

which returns the first (0) and second (1) column in each block. Thus, with these three easy steps, we created a ribbon, which happens to represent the first column in our original data file. All that is left is to walk through the columns in 'ribbon.dat', and do the same thing again and again. Note that we specified the columns that we are going to use as 1:2:3. We will need this later, when we shift the next ribbon to a new position.

Then, let us suppose that we are a bit lazy, and we do not fancy following the prescription above. We can then resort to the recipe outlined a couple of days ago in connection with the pie charts and the bar graphs. If you haven't read those posts, you should now, for I am going to use everything mentioned in them.

Since the three steps above are repetitious, we will put them in a 'for' loop. As I discussed it earlier, the way to do this is to write the instructions in a separate file, and 'reread' them as many times as necessary. I.e., we will have a file, 'ribbon_r.gnu', which contains the following lines

A=A+1
set table 'rib.dat'
plot 'ribbon.dat' u 1:1
unset table

set table 'rib2.dat'
splot 'rib.dat' mat
unset table

r=rand(0); g=rand(0); b=rand(0)
set palette model RGB defined (0 r/cm g/cm b/cm, 1 r g b)
splot 'rib2.dat' every ::0::1 u (A-1+B*$1):2:3 w pm3d
if(A<C) reread

So, this is our 'for' loop, and this is where we call it, 'ribbon.gnu'

reset
filename="ribbon.dat"
mult=1.2; cm=2.5
A=0; B=0.3;
set xlabel 'x axis [a.u.]'; set ylabel 'y axis [a.u.]';
set ticslevel 0
unset key; unset colorbox
set tics out nomirror
set border 1+2+4+8+16+32+64+256+512 back
mini(x)=(x<0 mult:x="" mult="" x="">0?x*mult:x/mult)

set isosample 100, 3
set xrange [0:1]; set yrange [0:1]
set table 'bg.dat'
splot x
unset table; set xrange [*:*]; set yrange [*:*]

splot filename mat
C=GPVAL_DATA_X_MAX+1
ymax = GPVAL_DATA_Y_MAX+1
zmin = mini(GPVAL_DATA_Z_MIN)
zmax = maxi(GPVAL_DATA_Z_MAX)
set xrange [-0.5:C-0.5]
set yrange [0:ymax]
set zrange [zmin:zmax]
set cbrange [0:1]

set multiplot
set palette model RGB defined (0 0.3 0.3 0.85, 1 0.9 0.9 0.95)
splot 'bg.dat' u ($1*C-0.5):($2*ymax):(zmin):3 w pm3d, \
'' u ($1*C-0.5):(ymax):(1.9999*$2*(zmax-zmin)+zmin):3 w pm3d, \
'' u (-0.5):($1*ymax):(1.9999*$2*(zmax-zmin)+zmin):(0) w pm3d
set cbrange [zmin:zmax]

unset border; unset tics; unset xlabel; unset ylabel; unset zlabel
l 'ribbon_r.gnu'
unset multiplot

This looks formidable at first, but we can easily decipher all. First, in our 'for' loop, in 'ribbon_r.gnu', we simply wrote down the three steps required to created one ribbon, we re-set the colour palette, and call the loop as many times as many columns there are (C).

In the main script, in 'ribbon.gnu', we first set up the main properties of the figure, and we define the width (B) of the ribbons, the multiplier (mult), which determines by how much the figure is bigger, than the minimum and maximum of the data points in 'ribbon.dat', and we also define 'cm', which will determine how "deep" the colour modulation of the ribbons is. By setting this to a number close to 1, we get a more or less homogeneous colour, while setting it to something large, we get a wildly modulated colour palette for the ribbons. Note the two helper function, 'mini' and 'maxi', which are used to set the zrange, and which make use of the value of 'mult'. The next couple of lines produces the background, which you can skip, if you do not want a very fancy graph. Then we make a dummy plot, just to determine the various ranges. You can read about this in my last but one post. Immediately after the multiplot, we plot 'bg.dat', i.e., the background of the figure. After this we call 'ribbon_r.gnu', and finally, escape from the multiplot.

A couple of notes: in the 'for' loop, we generated the colours by picking three random numbers. This means that your plot will have different colours after successive calls of 'ribbon.gnu'. You can change this, if you want to have some definite colouring scheme.

As I have already pointed out, the colour modulation of the ribbons is given by the variable 'cm', at the beginning of 'ribbon.gnu', while the size of the figure with respect to the range of the values in 'ribbon.dat' is given by 'mult'. If you set it to something smaller, than 1, be prepared for some clipping of the data!

Finally, these two scripts will produce the graphs automatically for you, the only thing you have to set is the name of the file, and, of course, you have to have the data in the format given at the very beginning of my previous post, i.e., the data points must be arranged in a matrix.

Tomorrow (or whenever I have time), I will show what else this script can be used for, with some small modification. So long!

Making a ribbon chart in gnuplot

While I am not a vary big fan of this kind of chart, I admit that on occasion, it might lend a refreshing new look to a graph. Besides, we can use it for something else, but more on that later. So, we will set to make this graph

from a data file similar to this

0.857354 0.943516 0.546751 -0.588053 -0.548767
0.551386 0.896607 0.163013 -0.166411 -0.442178
0.243002 0.310189 0.465029 0.0790944 0.378762
-0.24 -0.0625021 0.226812 0.48102 0.451281
-0.422674 -0.444781 0.314025 0.626378 0.822247
-0.513352 -0.893642 0.220384 0.800136 1.1908
-0.660823 -0.513169 0.316238 1.08866 1.25814
-0.0962106 -0.130051 0.0965893 0.944557 1.01623
-0.0152513 -0.0945605 0.468029 0.240179 0.556154
0.489453 0.726084 0.398192 -0.162081 -0.167408
0.876023 0.997108 0.139124 -0.281058 -0.581462
0.945061 1.19238 0.359603 -0.706723 -0.687105
0.860648 0.907059 0.217838 -0.540168 -0.592025
0.901743 1.02898 0.242619 -0.337589 -0.584658

which we will call ribbon.dat. There is an easy way to make this graph, and this solution requires an external script. Tomorrow, I will show a solution, if a bit convoluted, entirely in gnuplot, so that even those of you, who do not fancy a gawk one-liner, can make a similar figure.

The basic idea is to take the columns of ribbon.dat one by one, and turn them into a surface plot, whose derivative in the x direction is zero, while it takes the successive values of the column along the y axis. The gawk one-liner that does this transformation for us is given here

#!/bin/bash
gawk -v c=$2 '{printf "0 %d %g\n1 %d %g\n\n", NR-1, $c, NR-1, $c}' $1

I.e, we take the column designated by the value of 'c', and instead of listing the members, we print out six times the length of the column numbers, of the form

0 0 0.857354
1 0 0.857354

0 1 0.551386
1 1 0.551386

0 2 0.243002
1 2 0.243002

0 3 -0.24
1 3 -0.24

0 4 -0.422674
1 4 -0.422674
...

(If you look carefully, you will notice that this is nothing but the first four elements in the first column.) In this list, the first column is the 'x' value, the second column is the 'y' value, and the third is the 'z' value. We repeat all 'z' values twice, once with 0, and once with 1 'x' value. By plotting this, we will produce a ribbon of width 1. All that remains is to call the script multiple times, and change the colouring accordingly. With these in mind, our gnuplot script reads as

reset
unset key; unset colorbox
set tics out nomirror
set border 1+2+4+8+16+32+64+256+512 back
a=0.0
b=0.3
C=5.0

set xrange [-0.5:C-0.5]; set yrange [0:14]; set zrange [-2:2]; set cbrange [-2:2]
C=C+1.0
set xlabel 'x axis [a.u.]'; set ylabel 'y axis [a.u.]';
set ticslevel 0

set table 'bg.dat'
splot x
unset table

set multiplot
set palette model RGB defined (0 0 0 0.8, 1 0.9 0.9 0.9)
splot 'bg.dat' u 1:2:(-2):3 w pm3d, '' u 1:(14):($2/14*4-2):3 w pm3d, '' u (-0.5):2:($1-2):(-0.5) w pm3d

unset border; unset tics; unset xlabel; unset ylabel; unset zlabel
set palette model HSV function 1-a/C, 1-gray, 1-a/C
sp '<./ribbon.sh ribbon.dat 1' u (a+b*$1):2:3 w pm3d
a=a+1
set palette model HSV function 1-a/C, 1-gray, 1-a/C
sp '<./ribbon.sh ribbon.dat 2' u (a+b*$1):2:3 w pm3d
a=a+1
set palette model HSV function 1-a/C, 1-gray, 1-a/C
sp '<./ribbon.sh ribbon.dat 3' u (a+b*$1):2:3 w pm3d
a=a+1
set palette model HSV function 1-a/C, 1-gray, 1-a/C
sp '<./ribbon.sh ribbon.dat 4' u (a+b*$1):2:3 w pm3d
a=a+1
set palette model HSV function 1-a/C, 1-gray, 1-a/C
sp '<./ribbon.sh ribbon.dat 5' u (a+b*$1):2:3 w pm3d
unset multiplot

Here, 'b' is the width of the ribbons, 'C' is the number of ribbons we want to plot, and 'a' is just a variable for convenience. If you do not want to have the fancy background, you can skip lines starting with 'set table' to 'unset border'. Of course, if you do this, then you have got to move the line 'unset border;...' to after the first plot. In the rest of the script, we plot the columns, one by one, and also change the colour palette, so that the ribbons will have different colours. You can skip those lines, too, if you are satisfied with one colour for all the ribbons. I should also point out that the repeated call to the script can be made automatic, if you put those three lines into another gnuplot script, and call that script via reread. I discussed how to this this in my last two posts, so if you are interested in that, you should also read those. In those posts, you will also find hints as to how to avoid having to specify the various ranges by hand, and how to get the required values with the help of gnuplot. It should not be hard to implement those things in this script, so it is easy to make a "self-running" ribbon-plot-maker.

Well, this was about the easy way to make ribbon charts. In my next post, I will show how to do this in gnuplot, without having to rely on an external script. Till then!

Plotting a matrix with bargraphs

In some cases, it is common practice to plot a matrix with bargraphs, instead of a colour map, or something similar. For instance, in physics, people usually plot the density matrix (if it does not contain too many elements) in this way. I have already discussed how bargraphs can be generated, but that method called an external gawk script, which had the advantage that we hadn't got to know the number of elements in advance. Yesterday, I showed how we can avoid the use of an external script, provided we know how many points we want to plot. But could we make something better, and produce a plot entirely with gnuplot, and without knowing how many, and of what magnitude the elements are? We could, by making use of some gnuplot variables. This is what I am going to discuss today. Since I will build upon the tricks that we exploited yesterday, I would recommend that you read that post, if you haven't already done so.

Let us suppose that we have the following data file, called 'bar_matrix.dat', with 4 by 4 real numbers

1 0.5 2.2 0.5
1.2 0.6 -2.4 0.2
0.1 0.5 -1.8 -1.5
1.2 0.3 0.6 1.9

and we want to generate a plot similar to this:

Following the recipe from yesterday, we have to read out the numbers in the matrix, one by one, and plot the cuboids one by one. In order to give the impression of a 3D object, the three visible sides of the cuboids must be painted with different shades of the same colour, but that should not be a problem, we have already seen how to do that. However, there is one difficulty here: we will have to use multiplot, which means that in order to keep the relative size of the cuboids, we have to fix all three ranges, xrange, yrange, and zrange. In addition, determining the xrange and yrange serves not only some aesthetic purpose, but they will also control our 'for' cycle. So, what can we do to crack this nut?

In order to answer the question, we have got to think about how gnuplot plots: first, if no ranges are specified, data are read in, then based on certain values, the ranges are calculated. So, if we issue

plot 'something.dat' using 1:2

the first, and second columns of 'something.dat' will be read, and based on the minimum and maximum values of the first column, the xrange is calculated, and likewise for the yrange. But then these values are stored as a special variable in gnuplot. So, if you are interested in the minimum and maximum of your xrange, you can do

print GPVAL_DATA_Y_MIN
print GPVAL_DATA_Y_MAX

You can also print out all variables, both user-defined and internal, by typing

show variables all

Now, this means that we will have access to some properties of our data file, provided that the data file has been plotted at least once. This is what we will use in the following script, called 'bar_matrix.gnu'.

reset
filename="bar_matrix.dat"
w=0.4
FIT_LIMIT=1e-8
mult=1.05; cd=1.4

f(x,a)=(abs(x-a)<0.5?d:0)
R(x)=abs(2*x-0.5); G(x)=sin(x*pi); B(x)=cos(x*pi/2.0)

set view 60, 20
set parametric; set isosample 2, 2
unset border; unset tics; unset key; unset colorbox
set ticslevel 0
set urange [0:1]; set vrange [0:1]

splot 'bar_matrix.dat' mat
set zrange [mult*GPVAL_DATA_Z_MIN:mult*GPVAL_DATA_Z_MAX]
Y=GPVAL_DATA_Y_MAX+1
X=GPVAL_DATA_X_MAX+1

set xrange [1-w:X+2*w]

set multiplot

C=0; xx=1; yy=Y
call 'bar_matrix_r.gnu'

set palette model RGB functions 0.9, 0.9,0.95
splot -w+u*(X+3*w), -w+v*(Y+w), 0 w pm3d

C=1; xx=1; yy=Y
call 'bar_matrix_r.gnu'
unset multiplot

In addition to this file, we will need two more. Remember, in order to plot a matrix, we have to use two 'for' cycles, each requiring an external gnuplot script. So, we also have 'bar_matrix_r.gnu'

yy=Y
call 'bar_matrix_r2.gnu'
xx=xx+1
if(xx<X+1) reread

and 'bar_matrix_r2.gnu'

unset parametric
yy=yy-1
d=0.3
set yrange [*:*]
fit [0:Y] f(x,yy) filename u 0:xx via d
set yrange [-w:Y+2*w]
set parametric
r=R(yy/Y); g=G(yy/Y); b=B(yy/Y)
if(C==0 && d<0) \
set palette defined (0 r g b, 1 r g b);\
splot w*u+xx, w*v+yy, d w pm3d;\
r=r/cd; g=g/cd; b=b/cd;\
set palette defined (0 r g b, 1 r g b);\
splot w*u+xx, yy, v*d w pm3d;\
r=r/cd; g=g/cd; b=b/cd;\
set palette defined (0 r g b, 1 r g b);\
splot w+xx, yy+w*u, v*d w pm3d;

if(C==1 && d>0) \
set palette defined (0 r g b, 1 r g b);\
splot w*u+xx, yy+w*v, d w pm3d;\
r=r/cd; g=g/cd; b=b/cd;\
set palette defined (0 r g b, 1 r g b);\
splot w*u+xx, yy, v*d w pm3d;\
r=r/cd; g=g/cd; b=b/cd;\
set palette defined (0 r g b, 1 r g b);\
splot w+xx, yy+w*u, v*d w pm3d;

if(yy>0) reread

Having seen the scripts, let us discuss what the three files do. The beginning of 'bar_matrix.gnu' should be familiar by now, there is nothing new in it. The first interesting thing happens where we do a dummy plot of the matrix, just to extract the number of rows and columns, and the range of the data values. We use GPVAL_DATA_Z_MIN and GPVAL_DATA_Z_MAX to set the common zrange of all consequent figures, and GPVAL_DATA_X_MAX and GPVAL_DATA_Y_MAX to determine the number of iterations (and the xrange, and yrange, of course). Note that we use a multiplier, 'mult', defined at the beginning of the file, so that the top and bottom of the plot will definitely not be clipped. Then we clear our plot by setting the multiplot, and calling 'bar_matrix_r.gnu'. This file controls the 'for' cycles over rows. It does nothing, but re-sets the value of the inner 'for' cycle, increments its own variable, and calls 'bar_matrix_r2.gnu'. In this second 'for' loop is where the actual plotting takes place. In that file, we call our fitting routine, set the palette, and plot the 3 sides of the cuboids. Note that we want to paint the sides in different colours, so after plotting the top, we produce a darker shade by dividing all RGB values by 'cd', and after plotting the front side, we reduce the RGB values once more, to draw the right hand side.

Now, this script contains an 'if' statement, with the variable 'C' and 'd'. The reason for this is that we call 'bar_matrix_r.gnu' twice: first we plot all bars that represent a negative value, then draw the z=0 plane, and finally plot the positive values. This is what happens in the last couple of lines of 'bar_matrix.gnu'. If you are certain that your matrix contains positive values only, you can get rid of the 'if' statement, and also of the first call of 'bar_matrix_r.gnu'.
If you now call 'bar_matrix.gnu' from gnuplot, you will get the figure at the beginning of this plot. You can then add labels and so on, as you wish. I would also like to point out that human intervention is required only at the very beginning of 'bar_matrix.gnu', where we specify the name of the file that we want to plot. Otherwise, everything is automatic, and will just happen by the stroke of the key. You should also keep in mind that we had a dummy plot, so if you want to plot into a file, you have got to set the terminal after this plot, lest you should not have the dummy plot in your file.

Pie charts - entirely in gnuplot!!!

Some time ago, I discussed various ways to produce a pie chart in gnuplot. I have looked at the search terms that bring people to my blog, and it seems that this topic is one of the more popular, so I thought that it might be worthwhile to explore the issue more.

Since I am a Linux fan, I am quite fine with calling various scripts from gnuplot: pipes are rather convenient, and I can write a small script which does the data processing. But I see the downside of it, too: these solutions (while you can make it work under windows) will require extra steps, if not run under linux. I also understand that you might not want to delve into the nuances of gawk, for instance. So, I was wondering whether we could do everything in gnuplot, without relying on something external. And the answer is, yes, we can! (Otherwise, I wouldn't be writing this post at all...) What we are going to do is probably the dirtiest of hacks, for we will use a function of gnuplot, which was never intended to be used in such a ramshackle way. But we want to produce graphs, and this is not a coding beauty contest, after all!

If you recall, we had a data file with two columns: one designating years, the other one containing the values belonging to those particular years. Something like this, which we will call pie.dat:

1989 0.1
1990 0.2
1991 0.2
1992 0.05
1993 0.15
1994 0.3

You might also recall that the way in which we produced the pie chart was to plot arcs of a circle, the parameters determined by the second column. Now, the problem was (and this is why we had to use an foreign script) that those parameters cannot be set at run-time, so to speak, we had to hard-wire them into the gnuplot script. So, the question is really how we can access individual values of a data file, say, the 5th number in the 2nd column, and then do this repeatedly. The snag is that gnuplot hasn't a dedicated function to perform this task, so we have to look for a function that is dedicated to something else. Since there are not too many gnuplot functions that operate on a file, we can easily find the one that "returns" a value. We will use the fit function, and use the fitting parameter as a means of returning the sought-after value from the file. I know, I know! This is the ultimate abuse of 'fit', and we shouldn't do this at all! But what the heck! You haven't got to tell anyone how you produced that bloody figure!

Now, we have to find a proper fitting function. Remember, we want to pick the value of a particular number in the second column, say. Well, being a physicist, I would say that the Dirac-delta will do the job. However, we have to soften our stance a bit, and use something that is gnuplot-friendlier. For better or worse, I will choose the following function

f(x,a)=(x>a-0.5?(x<a+0.5?b:0):0)

You can recognise our old friend, the ternary operator, making its appearance twice in the definition of f(x,a). So, we first check, if the value 'x' is larger, than a-0.5. If so, we check whether it is smaller, than a+0.5. If this condition is fulfilled, we assign the value 'b', otherwise 0. By plotting it, you can see it for yourself that this function is a rectangle of height 'b' and width 1, centred on 'a', and 0 everywhere else. 'b' is going to be our fit parameter. Now, if you fit this function as

fit [0:7] f(x,2) 'pie.dat' u 0:2 via b
print b

then 0.2 will be printed. But that is the value of the 3rd number in the second column of pie.dat! (The rows are numbered starting with 0, that is why 2 means the 3rd row.) Since the value 'b' can be used in any subsequent expressions, definitions etc. as a number, we have found a way to extract any single number from any data file.

In order to proceed, we have to find a method to step through the rows of a column, one by one. For this purpose, we will use the reread command of gnuplot. You can learn the basic idea by issuing ?reread and ?if. 'reread' just repeatedly reads the file invoked by the last load command, and we can use 'if' to set some criterion as to how many times this repeated loading should take place. This is a primitive 'for' cycle, but it will do. (In gnuplot 4.3, the option 'for' was introduced in the 'plot' command, but that would not do too much good here, for we still have got to figure out the actual numbers.)

With these in mind, we write the following two scripts, the first of which named pie.gnu

reset
os=1.3
FIT_LIMIT=1e-8
L=6.0
f(x,a)=(x>a-0.5?(x<a+0.5?b:0):0)
r(x)=abs(2*x-0.5); g(x)=sin(x*pi); b(x)=cos(x*pi/2.0)

set view 30, 20
set parametric
set isosample 2, 2
unset border
unset tics
unset key
set ticslevel 0
unset colorbox
set urange [0:1]
set vrange [0:1]
set xrange [-2:2]
set yrange [-2:2]
set zrange [0:3]

A=0.0; D=0.0
set multiplot
# First, we draw the 'box' around the plotting volume
set palette model RGB functions 0.9, 0.9,0.95
splot -2+4*u, -2+4*v, 0 w pm3d
set palette model RGB functions 0.8, 0.8, 0.85

splot cos(u*2*pi)*v, sin(u*2*pi)*v, 0 w pm3d

call 'pie_r.gnu'
unset multiplot

while the second one named 'pie_r.gnu'

unset parametric
b=0.3
set yrange [*:*]
fit [0:L] f(x,D) 'pie.dat' u 0:2 via b
B=b
fit [0:L] f(x,D) 'pie.dat' u 0:1 via b
D=D+1.0
set palette model RGB functions r(D/L), g(D/L), b(D/L)
set parametric
set yrange [-2:2]
set urange [A:A+B]
set label 1 "%g", b at os*cos(2*pi*(A+B/2.0)), os*sin(2*pi*(A+B/2.0)), 0.2 cent
splot cos(u*2*pi)*v, sin(u*2*pi)*v, 0.2 w pm3d
A=A+B
if(D<L) reread

Now, let us see what is happening here. In the first script, 'os' will be used to place the labels later on. The gnuplot variable FIT_LIMIT is set to that value, so that the fit values are more accurate. It might be necessary to change it, if your labels are not what you expect. For the arcs, it should not really matter, because the default value is going to be accurate enough for any practical plots. 'L' is the number of data that we have (note that data are numbered starting with 0). Next, we define 4 functions. The first one is our fit function, the other three are used to colour the arcs. You can change these, if you are not satisfied with the colour scheme that you get. Any three functions will do, which are defined on [0:1], and return with a value in [0:1]. You can read about this in the post in which I discussed phonged surfaces, sometime in late May.

The next couple of lines set up the various ranges of our plot. I wrote about this in my first pie chart post, and the lines giving the background of the plot should also be familiar from that post. 'A' and 'D' are our control variables that we manipulate in 'pie_r.gnu'. We then draw the shadow of the pie, and finally, call 'pie_r.gnu'.

Let us take a look at 'pie_r.gnu'. First, we extract the value in the second column, and then in the first one. We will use this latter one to produce the label. Note that we have got to unset the parametric plot, otherwise, the fitting function would not work. Also note that we re-set the yrange. This is necessary, because the actual plot is in the [-2:2] range, while the fit is on values around 1990. Then we increment the value 'D' (this is the ordinal number of the row that we are currently processing), and re-set our palette, using the three functions, r(x), g(x), and b(x) that we defined in 'pie.gnu'. Then, using the extracted values, we set the range of the parametric plot, and define the label, and plot the arc. Finally, we check, if we have called the script enough times. Loading 'pie.gnu' will produce the following graph:

A couple of comments about this script: as I already mentioned, you can change the colour scheme by using various functions. However, if you are really lazy, you can simply generate three random numbers by

r=rand(0); g=rand(0); b=rand(0)

and assign these values to the next palette.

Second, you can easily implement an "explosion", by shifting one or several of the arcs by shift*cos(2*pi*(A+B/2)) and shift*sin(2*pi*(A+B/2)), based on some condition that you set. It should not be hard, either, to plot a real 3D pie char based on the scheme that I outlined about two months ago.

Third, if the sum of your numbers is not normalised to one, you can easily fix it by adding an extra loop to your script: if you have a script called 'pie_r2.dat', containing

unset parametric
b=0.3
set yrange [*:*]
fit [0:L] f(x,D) 'pie.dat' u 0:2 via b
D=D+1.0
G=G+b
if(D<L) reread

and call this in 'pie.gnu' immediately before 'pie_r.gnu', then 'G' will just be the sum of all numbers in column 2, and you can use this to normalise the numbers when you call 'pie_r.gnu'.

And last, the only thing you need in advance is the number of records you want to process, 'L'. This is the only thing you have to set by hand, all the rest is automatic. If you want to learn how to avoid this small "difficulty", you should read the post on the 2nd of August.

I hope that this script can convince people that we haven't got to rely on any outside tool, and can still produce a decent pie chart in gnuplot.I also feel that this approach is, in some sense, much cleaner, than the one that was based on gawk. Some people will probably contest this statement, but doing everything in gnuplot ensures that we retain the platform independence.

Adding shadow to the key

I think, it changes the impression a graph can make by quite a lot, if it looks sort of three-dimensional. Even a tiny twist to the boring 2D lookout can make a difference, though it doesn't give any new information to the reader. However, in a presentation, it is almost expected that one produces "exciting" graphs. Today, we will discuss an easy way to add a slight shadow to the key on a graph, as if it were lifted from the plane of the curves. At the end of our exercise, we will have the following figure

I believe the steps are quite straightforward, so I won't spend too much time on explaining every detail. Here is the script that we need

reset

xl=0; xh=1; yl=-1; yh=1;
eps=0.01;
rx=0.6; ry=0.8; kw=0.35; kh=0.15
lh=0.06; al=0.1

key1="First function"
key2="Second function"

set table 'shadowkey.dat'
splot [xl:xh] [yl:yh] x/(xh-xl)
unset table

set object 1 rect from graph rx,ry rto kw,kh fc rgb "#aaaaaa" fs solid 1.0 front lw 0
set object 2 rect from graph rx-eps,ry+eps rto kw,kh front fs empty
set label 1 at graph 1.1*al+rx, ry+2*lh key1 front
set label 2 at graph 1.1*al+rx, ry+lh key2 front
set arrow from graph rx, ry+2*lh rto al, 0 lt 1 lw 1.5 nohead front
set arrow from graph rx, ry+lh rto al, 0 lt 3 lw 1.5 nohead front

unset colorbox
unset key
set palette defined (0 "#8888ff", 1 "#ffffff")
plot [xl:xh] [yl:yh] 'shadowkey.dat' w ima, \
x*x*exp(-x) lw 1.5, cos(13*x)*exp(-3*x) lt 3 lw 1.5

I collected all variables at the beginning of the script, so that it will be easier to adapt it to any situations. The first four numbers define the xrange and the yrange. The next four numbers specify the position and the size of the key. We will have to make our own key, and the size of the key will depend on the particular terminal one uses. This is why it is handy to define them at the beginning of the script, so if you are not satisfied with the results, you can easily adjust both the position and the size. The next number (lh) gives the hight of one key line, while 'al' will be the length of the line which represents the curve. 'key1' and 'key2' are just two arbitrary strings, holding the text of the key.

The next three lines are necessary only if you want to draw the background gradient, but if you are happy with a white plot, you can skip this. I should also point out that the first four numbers that are needed for the xrange and yrange, are needed only because we have to "synchronise" the plot with its background. If you don't want the gradient, you can drop these definitions, and you haven't got to specify the range in the plot either. This might be useful, when you don't know the plotting range beforehand, and want to let gnuplot calculate it for you.

The next step is to draw two rectangles in the front of the graph: one gray, and one white. Note that the rectangle in gray has no boundary, i.e., that is set to zero width. Also note that we specify the coordinates in terms of the numbers we defined at the beginning, so both rectangles will change accordingly, if you change those numbers. This means that the white rectangle and its shadow will always be linked, given by the variable 'eps'. When we are done with the white rectangle, we can place the text of the keys, and the two lines representing our plots. For this purpose we draw two arrows without heads, and with the linetypes that we will use for plotting the curves.

The final step is to actually draw the curves. Before that, we plot our file, 'shadowkey.dat', so the graph will have a background gradient. If you skipped those three lines writing the data file to disc, you should replace the plotting command by

plot [xl:xh] [yl:yh] x*x*exp(-x) lw 1.5, cos(13*x)*exp(-3*x) lt 3 lw 1.5

It should go without saying that in this case, you can also drop the palette definition, for it will not be used. Well, this is it for today. I know that this was a simple trick, but we can't have something complicated every day!

Maps - Contour plots with labels

So, you have always wondered how on Earth one can make a real map with gnuplot. Well, there is a simple and a not-so-simple way to this. First, let us see the simple way. Since it is simple, this method won't have one of the features, isoline labelling, that the second one has. As we go along, the map will become more and more complicated, but I hope that I set the right pace, and it will be easy to follow.

A map is nothing but a colour-coded 3D plot, with the isolines attached to it. We will produce a simple map using the function

sin(1.3*x)*cos(0.9*y)+cos(.8*x)*sin(1.9*y)+cos(y*.2*x)

I don't think that this function has any particular meaning, but it looked quite all right, at least, as far as maps and isolines are concerned. If you have a matrix to plot, you can replace this function with that file. In principle, we could plot both the colour-coded map and the isolines from gnuplot, but we will have much greater flexibility, if we first direct the plots to a file, and then call the data from those files. One of the advantages of doing this is that in this way, we can make sure that the various plots have the same size. This requires a small overhead in terms of scripting, namely, we have got to issue the command

set table 'something.dat'
splot something
unset table

While this might appear superfluous, there are good reasons to do this. If we plot our function through the file 'something.dat', then we can use the 'with image' modifier of the plot command, and this means that the plot will be of the same size as would a normal 2D plot. Otherwise, if we use

set pm3d map
splot something with pm3d

the plot will actually be a bit smaller. The reason behind this is that in a 3D plot, we have to have some space for the 'z' axis, and even if we drop it in the map view, the space-holder for the 'z' axis is still there, therefore, gnuplot makes the whole plot a bit smaller. If, however, we plot the map through a file, we can use 'plot', in splot's stead, therefore, the 'z' axis will not appear anywhere in the processing of the plot.

After this interlude, let us see our first version of the map.

reset
f(x,y)=sin(1.3*x)*cos(.9*y)+cos(.8*x)*sin(1.9*y)+cos(y*.2*x)
set xrange [-5:5]
set yrange [-5:5]
set isosample 250, 250
set table 'test.dat'
splot f(x,y)
unset table

set contour base
set cntrparam level incremental -3, 0.5, 3
unset surface
set table 'cont.dat'
splot f(x,y)
unset table

reset
set xrange [-5:5]
set yrange [-5:5]
unset key
set palette rgbformulae 33,13,10
p 'test.dat' with image, 'cont.dat' w l lt -1 lw 1.5

In the first couple of lines, we plot the actual function into a file, called 'test.dat'. There will be some 25000 data points in this file, for there are 100 samples (default), and 250 isosamples. Having written these points into a file, we plot the contour. We just happen to know that the value of f(x,y) is between -3 and 3, so we set the contour levels at -3, -2.5...2.5, 3. When we are done with all this, we simply reset our plot, set the x and yrange, specify the palette that we want to use with the colouring, and call the plot command. For the actual data points, we use the 'with image' modifier, while for the contours, we increase the linewidth to 1.5, instead of using the default value of 1. We, thus, have just produced the following image.

Now, this was sort of standard, but the question inevitably comes up, whether we could do something more with this. When one has contour lines, one wants to label them, I presume. Petr Mikulik has a gawk script that will do this, in particular, it will place the appropriate labels next to the contour lines. I will discuss another approach here. The main difference with Petr Mikulik's method is that here we will put the labels on the contour lines, with some white space, of course.

First, it is quite instructive to look at the file containing the contour lines. In order not to cobble the plot, we will restrict it to the range [-5:0] (x) and [2:5], but it is really just for the sake of simplicity. With this modification, the file should read as

#Surface 0 of 1 surfaces

# Contour 0, label:        2
-0.482969  3.59036  2
-0.505051  3.58024  2
-0.509852  3.57831  2
-0.539895  3.56627  2
-0.555556  3.55994  2
-0.572136  3.55422  2
...

# Contour 1, label:      1.5
-1.11111  4.57693  1.5
-1.10874  4.57831  1.5
-1.0877  4.59036  1.5
-1.06617  4.60241  1.5
-1.06061  4.60546  1.5
-1.04273  4.61446  1.5
...

What this means is that the 0th contour line is drawn at z=2, and then the (x,y,z) values are listed. Of course, since we know that this particular line is at z=2, the last coordinate doesn't play any role, but it is listed all the same. When we plot this file, consecutive (x,y) points will be connected by a straight line segment. The next contour is at 1.5 etc. It might happen that one contour line is made up of several blocks, both for technical, and fundamental reasons. The technical reason is gnuplot's inner procedure of computing the an isoline, while the fundamental is simply that there might be several disconnected regions with the same isolevel. Anyway, if there are several blocks, they are separated by a blank line within the same contour.

In order to create the labels and the white spaces for them, we will use the following short script:

#!/bin/bash

gawk -v d=$2 -v w=$3 -v os=$4 'function abs(x) { return (x>=0?x:-x) }
    {
            if($0~/# Contour/) nr=0
            if(nr==int(os+w/2) && d==0) {i++; a[i]=$1; b[i]=$2; c[i]=$3;}
            if(abs(nr-os-w/2)>w/2 && d==1) print $0
            nr++
    }
    END {   if(d==0) {
                    for(j=1;j<=i;j++)
                    printf "set label %d \"%g\" at %g, %g centre front\n", j, c[j], a[j], b[j]
            }
    }' $1

This script operates on the data file of contour lines that we had before, and simply takes out a couple of points of the contours, while keeping one of the coordinates as the position of the labels. You can change the length of the white space by setting the 'w' argument, while the position of the white space can be modified by changing the 'os' argument. The first argument, 'd', determines what we want to do with the script: make the labels, or tweak the input file.
Now, we can call the script from our gnuplot script as follows:

reset
set xrange [-5:0]
set yrange [2:5]
set isosample 150, 150
set table 'test.dat'
splot sin(1.3*x)*cos(.9*y)+cos(.8*x)*sin(1.9*y)+cos(y*.2*x)
unset table
set cont base
set cntrparam level incremental -3, 0.5, 3
unset surf
set table 'cont.dat'
splot sin(1.3*x)*cos(0.9*y)+cos(.8*x)*sin(1.9*y)+cos(y*.2*x)
unset table
reset
set xrange [-5:0]
set yrange [2:5]
unset key
set palette rgbformulae 33,13,10
l '<./cont.sh cont.dat 0 15 0'
p 'test.dat' w ima, '<./cont.sh cont.dat 1 15 0' w l lt -1 lw 1.5

There are two more twists that we could add to the figures above. One is that the labels can be rotated in such a way that they are parallel to the curves. This we can easily achieve by adding two more lines to our script above: we take the first and last dropped coordinate, and calculate the angle that that line segment would make with the horizontal. Using this angle, we rotate the labels accordingly. With this modification, the script now becomes

#!/bin/bash

gawk -v d=$2 -v w=$3 -v os=$4 'function abs(x) { return (x>=0?x:-x) }
    {
            if($0~/# Contour/) nr=0
            if(nr==int(os+w/2) && d==0) {a[i]=$1; b[i]=$2; c[i]=$3;}
            if(nr==int(os+w/2)-1 && d==0) {i++; x = $1; y = $2;}
            if(nr==int(os+w/2)+1 && d==0) r[i]= 180.0*atan2(y-$2, x-$1)/3.14
            if(abs(nr-os-w/2)>w/2 && d==1) print $0
            nr++
    }
    END {   if(d==0) {
                    for(j=1;j<=i;j++)
                    printf "set label %d \"%g\" at %g, %g centre front rotate by %d\n", j, c[j], a[j], b[j], r[j]
            }
    }' $1

and the resulting graph is shown here:

Note that the only difference between this and the previous script is the following two lines

if(nr==int(os+w/2)-1 && d==0) {i++; x = $1; y = $2;}
if(nr==int(os+w/2)+1 && d==0) r[i]= 180.0*atan2(y-$2, x-$1)/3.14

where we calculate the appropriate angles.

The second twist is that it might make interpretation of the figure easier, if the contours are coloured, as are the corresponding labels. We can use the script above with some small modification. The key is to use the 'index' modifier. Recall that the contours are rendered in blocks, and with the help of the 'index' keyword, we can specify which block we want to plot. We, therefore, modify our script as follows:

#!/bin/bash

gawk -v d=$2 -v w=$3 -v os=$4 'function abs(x) { return (x>=0?x:-x) }
   {
           if($0~/# Contour/) nr=0
           if(nr==int(os+w/2) && (d%2)==0) {a[i]=$1; b[i]=$2; c[i]=$3;}
           if(nr==int(os+w/2)-1 && (d%2)==0) {i++; x = $1; y = $2;}
           if(nr==int(os+w/2)+1 && (d%2)==0) r[i]= 180.0*atan2(y-$2, x-$1)
           if(abs(nr-os-w/2)>w/2 && (d%2)==1) print $0
           nr++
   }
   END {   if(d==0) {
                   for(j=1;j<=i;j++)
                   printf "set label %d \"%g\" at %g, %g centre front rotate by %d tc lt %d \n", j, c[j], a[j], b[j], r[j], j

           }
           if(d==2) {
                   printf "plot \"test.dat\" w ima, \"cont.plt\" index 0 w l lt 1,\\\n"
                   for(j=2;j<i;j++) printf "\"\" index %d w l lt %d,\\\n", j-1, j
                   printf "\"\" index %d w l lt %d\n", i-1, i

           }
   }' $1

Calling the script produces the following graph:

Note that we had to call plot through a new temporary file, cont.plt, which should be produced by redirecting the output of

cont5.sh 2 15 0 > cont.plt

Grid with multiple colours

A couple of days ago, I very badly needed a graph on which all four axes were used, and in addition to that, I needed a grid on the graph. Now, the problem is, if you just plot a grid, it will be hard to follow which grid line corresponds to which axis. So, I decided to colour both the grid, and the axes, and use the same colour for corresponding grid lines and axes. Well, this is what I decided, but then it turned out to be a rather hard nut to crack. (In gnuplot 4.3, the situation is a bit better, I will discuss that tomorrow.) I have found a work-around, however, so if you are interested, you can find the details below. But let us see the graph first!

The standard way of defining a grid is a line similar to this:

set grid ytics lt 0 lw 1 lc rgb "#880000"

First, we define where we want to have the grid (everywhere, where there is an ytics), then the linetype, the line width, perhaps, and the colour. The problem is that when you issue this command multiple times, for some funny reason, only the last colour setting will take effect, i.e., if after the line above, you happened to have

set grid y2tics lt 0 lw 1 lc rgb "#008800"

then gnuplot would place the grid lines at y2tics all right, but the grid lines at ytics would also be green, and this is not what we want. The way out of this difficulty is to trick gnuplot into thinking that we have more than one plot, and re-set the grid before each new plot. Of course, we don't actually want to plot anything after the first one, therefore, we will just plot 1/0. It is a vicious way to deceive gnuplot, but it does the trick. Now, here is the script

reset
h=4.136e-15 # eV s
hbar=6.57e-16 # eV s
c=3e17  # nm/s

set multiplot
unset key
set xlabel 'Wavelength [nm]' tc rgb "#0000ff"
set x2label 'Temperature [K]'
set ylabel 'Energy [eV]' tc rgb "#880000"
set y2label 'dE/d{/Symbol l} [meV/nm]' tc rgb "#008800"
set xrange [200:2000]
set x2range [0:10]
set yrange [0:6]
set y2range [1:-7]
set y2tics 1,-1,-7
set ytics nomirror tc rgb "#880000"
set y2tics tc rgb "#008800"
set xtics nomirror tc rgb "#0000ff"
set x2tics
set mxtics 2
set mytics 2
set grid xtics lt 0 lw 1 lc rgb "#0000ff"

plot h*c/x w l lc rgb "#880000", -1000*h*c/x/x axes x1y2 w l lc rgb "#008800"

unset grid
set xlabel " "
set xtics format " "
set x2tics format " "
set x2label " "
set ylabel " "
set y2label " "
set grid x2tics lt 0 lw 1 lc rgb "#000000"
plot 1/0

unset grid
set grid ytics lt 0 lw 1 lc rgb "#880000"
plot 1/0

unset grid
set grid y2tics lt 0 lw 1 lc rgb "#008800"
plot 1/0

unset multiplot

First, we define a couple of constants, and then set up the axis labels. Note that when doing so, we assign a colour to each axis, so that we will know which axis the plot belongs to. Of course, we could do this with arrows, but since we want to have coloured grids, it is better to do it in this way. Next, we set the various ranges, and then set up the first grid line, which will be drawn at every xtics, and in blue. (Blue was the label colour of x.) After plotting the curves, we unset the grid, and we also take off the labels and ticlabels. Note that when doing so, we, in fact, re-set the labels, but with an empty string. The reason for this is that, if we simply take off the labels, then the margins of the graph will change, and when we plot our next graph (which is empty, but we still need the grid), it would be of a different size, and would not align with our original plot. Having re-labelled the graph, we re-set the grid, this time putting the lines on x2tics in black, then on ytics in a dark shade of red, and finally, on y2tics, in dark green. At the very end, we unset the multiplot, i.e., escape from the whole plot.
This is a bit dirty, but if you are pressed for it, it should still be OK. Next time, I will show the easy route, provided you have gnuplot 4.3.