The steps of patching gnuplot are described on gnuplot's main web page. There are a number of patches uploaded to gnuplot's patch tracker, on which quite a few new features, still in the development phase, are published. It is really worthwhile to try them out, first, to provide feedback as to what is useful and what is not, and second, to help the developers to find bugs and other glitches, like what the syntax of a command should be and so on.
Our patch is related to an old debate as to what gnuplot really is. At many a place, you will find the statement that "gnuplot is a plotting utility, not a statistical analysis package". I have nothing against this statement, however, when saying so, we have to tell what we mean by plotting. So, is plotting just placing a thousand dots at positions that represent our data? Or do we want more? E.g., throwing out data points that are unreasonably far from the mean. Or showing the mean, and the standard deviation? Or calling the reader's attentional to some special points, like the minimum or the maximum in a data set? And many similar things. I believe, plotting requires much more, than just showing the measurement data: a plot makes sense only, if we can point out what is to be pointed out. By the way, fitting falls into this category, and fitting has been an integral part of gnuplot for ages. The point being that the original statement (gnuplot is a plotting utility, not a statistical analysis package) has been wrong for a long time.
The patch that I mentioned above was announced yesterday on the gnuplot development mailing list and you can find the patch for the source and the
documentation on patch tracker. I have put a couple of examples on my gnuplot web site under patch. You can also find the full documentation.
I would like to ask you, if you feel crafty and you can, download the patch, and try it, and let us know whether you find it useful, what else, do you think, we could do with it and so on. It would really help the development. Once the patch makes it to the main code, I will discuss various option on these pages.
Just to wet your appetite, here is a figure that you could very easily make with the new patch. (You can find the code on my web site.)
Many cheers,
Zoltán
times
Dear Zoltán,
ReplyDeletethank you very much for this patch (and also thanks to Philipp Janert). I agree with you that these features (min, max, mean, etc.) are not introducing statistics into Gnuplot, but merely provide a clean and simple way to help representing scientific data.
I personally needed these features in too many occasions and I had to go through messy tricks to get the results I wanted.
I look forward to see this patch included in the official distribution of Gnuplot.
Thank you again,
Matteo Tommasini
Dear Matteo,
ReplyDeleteMany thanks for the feedback. We have implemented some improvements (e.g., matrix, and iterations are now supported by stats), and hopefully it will make it to the main branch sometime soon.
Cheers,
Zoltán
Dear Zoltán,
ReplyDeletethis is great news :-)
I will stay tuned...
All the best,
Matteo
Hello everybody,
ReplyDeleteI already posted this on SF, but I guess you don't mind if I repeat it here:
First of all I want to say that I really appreciate the stats patch. I really
needed this functionallity, too. I hope it will make it to the public version.
Now, that I have worked some time with stats, I would like to suggest
another way of invoking the stats functionallity. Because, I have not yet
read the source code, I don't know if it is possible.
Instead of invoking stats before one or serveral data file(s) are plotted
and storing each stats results in tons of variables with different prefix,
isn't it possible to enhance the plot command by a "stat" tag? May be an
example explains more:
Instead of
stats "file1" noout var=file1_
stats "file2" noout var=file2_
plot "file1" u 1:($2/file1_max_y), \
"file2" u 1:($2/file2_max_y)
... this will give a plot with a relative y-axis (0 to 1)
One can easily imagine the work overhead if you want to compare several
files in such way.
For this reason I propose something like that as additional
functionallity:
plot "file1" u 1:($2/$max_y) stats statsusing 1:2, \
"file2" u 1:($2/$max_y) stats statsusing 1:2
Of course the syntax is not perfect, but hopefully the idea can be seen.
For the standard usage statsusing (or other {datafile-modifiers}) may be
omitted. In genereall there must be at least a way to have a different
using than that of the plot command, because the latter one will contain
"variables" like $max_y. I have no idea if the return values like $max_y is
a good way to handle the stats results.
Anyway, the stats tag has to retrieve the information and set variables
before the actual ploting starts.
I hope you like this idea. Looking forward for comments on that.
Best regards.
Hello Andreas,
ReplyDeleteFirst of all, many thanks for the feedback! What you suggest is a sound idea, but there are a few obstacles in the way.
First, we wanted an implementation that doesn't interfere with any existing code, which means that it had to be independent of the plot command, and the like.
Second, if we go your way, we would have re-written the command parsing mechanism, which wouldn't have been a piece of cake...
Thirdly, and perhaps, most importantly, in your scheme, statistical properties can't be used for anything else, but plotting. What I had in mind at the very beginning is putting an arrow at the place of the maximum, and adding a label, showing the value of the maximum. In your scenario this cannot be done.
I don't think that assigning the values to variables is such a hassle, by the way. If the many new variables disturb you, you can do the following
stats 'foo' using 1:2 var pre "baz_"
max1 = baz_max
unset baz_*
which will leave you with one variable, max1, and nothing else. Note that the syntax of stats is a bit different to the original version. Also note the unset command which can take a wild char. There is a recent patch by Philipp Janert that can do this. (By the way, the reason for changing the syntax a bit is that in this version, the prefix is a string, while originally it was a bareword. This latter one can't be scripted, while the string can.)
Well, these were our arguments for the syntax of stats. I will try to roll out the new version in the coming days.
Cheers,
Zoltán