Matrex 2.1 released

Matrex 2.1 has been released yesterday.

Version 2.0 was revolutionary because it introduced the concept of client/server; version 2.1 is more an evolution because its purpose is to help the user having full control on projects and on the system.

In detail, these are the big changes:

1. The grid/tables have now a real row header, making easier to work with them and giving them a more professional look.
2. The script editor, which was very simple, has become a small but complete IDE to write and test scripts. With it the user can write its own functions and can test them before he uses them in his projects.
3. For someone that uses Matrex for the first time it is very important to find out immediately how it works. To make it possible, Matrex has now a help button in every window. Here is an example:
   
   

The other changes are less visible, but still important:

1. Before 2.1 the user could change all the function templates, included the ones that come with the system. Now he can only change templates written by users.
2. Fixed permission problems in the installation under Windows, allowing to install under the Program Files directory.

Thanks to Braxton for building the Matrex setup

file for MacOSX. Here is a screenshot of Matrex under MacOSX:

In the next version I will try to change the grid row header buttons under MacOSX to make them rectangular.

Matrex 2.1 is downloadable from here.

The new script editor

Matrex has always allowed to use a script language (Jython, Groovy, JRuby...) to write the code of the functions.
But until now the support was limited to the following functionalities:

• A button in the function template editor to associate a script to a function template.
• A simple editor to write the script.

From version 2.1, Matrex will have a small IDE to write scripts:

The IDE has:

• An editor with a simple but effective syntax highlighting. For each script language defines its specific highlighting rules.
• A graphical test environment that allows to set up input matrices and parameters, with their content, and output matrices of the script. Once the matrices are set, the script can be executed and the result of the script can be checked in the output matrices.

In this way it is possible to edit the script and test it before it gets use as a Matrex function.

Matrex 2.1: status

I am working on version 2.1 of Matrex .

The work on version 2.0 has been completely dedicated to the conversion of Matrex from a pure standalone to a standalone + client/server architecture, so I had to postpone some improvements to the GUI and the implementation of new functions.

Now I can finally work to implement these changes.

The planned changes for version 2.1 are the following:

• Conversion of all the grid/tables (editors/viewers) so that they use the new table row header, which has shown to work without glitches and to be extremely efficient, even with large matrices and virtual tables. Already implemented.
• New Help button in every window/dialog. Already implemented.
• Conversion of the script editor to a simple script IDE, which allows to test the function scripts before they are added to the system, and has simple syntax coloring. Working on it.
• Add several new functions. I think Matrex has already all the functions needed by most of the people, but I think more specialized functions are need, so that every kind of user has what he needs. Will start soon.

This is the plan, but probably some more feature will be added to version 2.1. See the active tickets.

You can have a snapshot of version 2.1 from the subversion repository, which should always contain a working version.

If you want to help, you are welcome to join the project.

SWT Table: row header

I always used the SWT Table as grid component for my project Matrex.

The problem of this grid component is that it does not have a row header; and this is a real problem for an application like Matrex, that is an alternative to spreadsheet applications.

There are alternatives to the SWT Table that have a row header, but even if some of them are very good I personally prefer to use the original SWT Table.

So I tried the fixed first column horizontal scroll remaining columns SWT snippet, where the row header consists in a second SWT Table with only one column, syncronized with the main table.

This solution works fine in Windows, but has the following disadvantages:

• In Gnome/Linux scrolling the table up and down the first row can be partially hidden by the column header. This is not handled by the snippet, therefore from time to time the row header appears not synchronized with the table.
• The cells of the row header table are normal cells, not header cells (3D gray-button look). This can be fixed, but it looks like for a table of 10000 rows you need to create 10000 buttons!

So I have found a different solution, which uses a separate control for the row header, like in the snippet.

The row header control is a panel containing a vertical set of buttons, one for each visible row of the table.

You can see the panel on the left in the following picture (which is based on the SWT snippet create a table (columns, headers, lines)) :

In the picture, each button shows the index of the related row.

When the user scrolls the table vertically, the buttons texts change accordingly.

If part of the row is partially hidden by the column header, like in the picture, the related button is also partially hidden.

If the horizontal scrollbar appears, the row header changes its size accordingly.

In synthesis, the row header is always synchronized with the table.

You can see a demo of the row header in this video.

The table row header is just a single class, that can be tested using this test class.

The only complicated part of the class is the function getOffset, which solves the problem of the "partially hidden first row", returning the part of the first row that is hidden.

It uses the SWT Table function table.getItem([point]), which returns the row at the given point.

With this function it can know where is the limit between the first and second row, and therefore when the first row finishes. Knowing the height of each row (table.getItemHeigh), it calculates which part of the row is hidden.

I tested it with Windows and Linux (Ubuntu) and it works fine. It could be nice if someone can test it on MacOSX. There should be no problem, since it uses only documented functions in a standard way.

I will use it on all tables of the Matrex project.

Rowscope: view large files

I had a problem in the office: I had a server application that was generating a large log file every day (between 100 Megabytes and 1 Gigabytes) and, if something went wrong with the application, viewing this log file was very difficult.

I tried several file viewers, but none of them worked well for me. Some of them did not show the entire row, some of them did not show all rows, some of them was slow when searching.

So, I went back to use notepad, which worked generally very well, but used too many resources and when the file was over the 300/400 Megabytes it was using the whole CPU and memory of the server.

So I have decided to write a file viewer and to write it in Scala, also because I wanted to try this language. The new file viewer, called Rowscope, had to be lightweight and fast.

For this purpose, I used a trick. Instead of viewing the file and then search strings in it, the user must first enter the search string, then when he opens the file what he sees are the rows resulting from the search.

Then he can look what's around one or more of the found rows.

In this way, if the search string is good enough, the viewer shows only a few rows and therefore uses a limited amount of memory. Clearly it reads all rows, but if they don't match the search string they are discarded immediately and then garbage collected.

To avoid that the GUI blocks Rowscope uses multiple threads in a pipe (Scala actors) to do the real work, and the rows are passed to the GUI thread only when they need to be displayed.

With the correct settings for each actor (how many rows to read from the file before they get sent to the other actors, how many rows to send from one actor to the other in one shot), the application uses the correct amount of CPU and memory and remains reactive to the user input.

I was a little scared when I started to use it at work. Yes, theoretically it should work, but will I use it just because I wrote it or because it really works?

I can say that now I use it without even thinking I am doing it, like all the other tools I use in my daily job. The first search then view approach works.

If I don't know what to search I start it with an empty search string, just to have an idea of what the file contains. Rowscope shows only the first 1000 lines of the file. From this I find a good search string and I start to work.

I'm preparing a new version of Rowscope. If you miss some feature or have some good idea, please let me know.

Matrex 2.0 released

Finally the version 2.0 of Matrex has been released.

The big change is the new optional client/server architecture that allows several users to work together on a project or for a user to delegate the calculations to a server (in the picture we see 3 Matrex Desktops sharing a project on a server run in the same PC).

We already told everything about this feature in the previous articles in this blog (read for example the article showing the steps to follow to connect to a server).

But we have other changes that are really useful.

First of all, Matrex does not need a batch file to start anymore. Just double-click on the matrex_start.jar file and Matrex starts (if java is configured correctly).

This also removes the shell/command line that was always starting together with the GUI, making the application look unprofessional.

Second, Matrex gets installed on MacOSX as an application bundle (packaged by Braxton), that's why its name is terminating with .app.dmg.

Unfortunately the application bundle can only be installed on MacOSX 64 bits.

For the 32 bits version of MacOSX (Java 5) you will need to install the generic version, then, using a shell, go to the top Matrex directory, run:

java -XstartOnFirstThread -jar Java/matrex_start.jar

and follow the instructions to install the GUI library (check also the F.A.Q. about this).

Screencast: connect to server

I was not able to publish Matrex 2.0 until now, because the test took more time than I thought initially and because Braxton and me are trying to make it installable on MacOSX using an application bundle.

Hopefully in a few day the new version will be published.

Anyway, I made a screencast, published on ScreenToaster, on how to connect Matrex to a server on the same PC:

Screencasts and videos online

Matrex 2.0 Beta released

A beta of the client/server version of Matrex, 2.0, has been released.

The plan is to release the final version in a pair of weeks, so please test it!

To test the client/server connection:

• Install Matrex 2.0 beta.
• Follow the instructions contained in the server document (which is included in the installed documents) to install the Matrex Server and start it.
• Start Matrex Desktop and connect to the server following the Remote Projects section of the help content (press Ctrl-F1 in the Matrex Desktop).
• Create a project on the server or copy a local project to the server, as described in the help content, and play with it.

If something is not clear, please contact me.

If you find bugs please report them in the trackers.

Matrex 2.0: project integrity in a multi-user system

Now it is a long time the alpha version of Matrex 2.0 was published, but a beta is still not available. The reason is that the task to make the core objects (project, matrix, function...) thread safe is complex.

Until now Matrex was a single-user system. That does not guarantee against concurrency problems, but makes certainly things easier than on a multi-user system, like Matrex 2.0. In Matrex 2.0 a function or chart or presentation can potentially be changed by multiple users in the same time, which can lead to unpredictable results.

To avoid this, the work I'm doing now is to make the project object not only a repository of its items (matrices, functions...) but to give it the responsibility of its own integrity, and to this, the complete responsibility of adding, updating and deleting items.

In Matrex 2.0 the chart editor, for example, is not able to update a chart object directly. It tells the project how to change the chart object, and the project changes it, after it checked that the change is possible and does not break its own integrity: charts are dependent by matrices and it could be possible that the change involves adding a matrix to it that during the chart editing gets deleted by a different user; in this case the project rejects the update, the chart does not get change and the editor gets an error.

The project's methods are guarded, so that it is impossible to do incompatible changes in the same time. For example it is not possible to delete a matrix in the same time a chart is updated.

In other words the project in Matrex 2.0 looks a lot like an SQL server with transactions. The SQL server makes sure two SQL updates cannot take place on the same rows of a table in the same time and an SQL update (or a group of them) can be integrally rejected (rollback) if something goes wrong, leaving the database as it was.

A side effect of these changes is that, since everything is done through the project, the items become more or less read-only for the editors and the rest of the GUI, simplifying the system.

In terms of performance these changes should be in-influent. What is important for the system performance are not the items editing, but the matrices calculations that derive from it, which are much more complex and CPU consuming.

I'm now in the middle of the work of changing project, items, editors to make them work in this way. As soon as I am finished I will publish a beta.