No se me ocurre mejor acompañamiento para un proyecto como éste donde el conocimiento de los datos es crucial. Poca gente en España conoce los datos y procesos EIEL tan bien como ellos, una pequeña muestra es el gran trabajo que han liderado con gvSIG-EIEL. Si queréis saber más de IDEPo o simplemente hablar de iCarto, estaré disponible en el WTC al menos todo el día del jueves.

Is it something we could extrapolate from the data there?

Certainly, not the number of features developed or bug fixes. It is even barely possible to compare activity between projects, as there are a high variability in terms of changesets: some people could send several little changesets and others just 1 big change, some project could have a special policy which affect the results (i.e.: make a commit formatting the code accoring to the style rules and other with the changes), etc. Some people could even argue that the language they are written in affects the number of changes (GRASS is written in C, gvSIG in Java and QGIS in C++) due to the libraries available or the semantics of every language. So, is it possible to find out something? Well, in my opinion, we can trace at least the following:

• the internal evolution of a project.
• how a project is doing in terms of adding new blood.

 So, let’s make again the exercise of finding out what’s happening here:

GRASS

• It calls the atention the curve of activity in the project: growth by periods (2001-2004 and 2005-2007) with local maximums in 2004 and 2007. Our hypothesis was that it was due to the way the project works: the developers here make changes both in the trunk and in the branch of the product to release (be it 6.4 or 6.5) at the same time, with a lot of changesets moved between both the trunk and the branches (so doing heavy backporting). In a recently conversation with Markus Neteler, he has explained me better how they work and I guess the rhythm we see in the graphics is due to that.
• In terms of number of developers, GRASS has showed a continuous growth until 2008; since then, the number of regular developers stabilizes.

gvSIG

• gvSIG shows an incredible high period of activity during 2006-2008 (4500 changesets by year and most that 30 people involved!). To understand the Gauss bell of activity, is needed to know the background of the project: gvSIG development has been led by contract, which means that all activities (planning, development, testing, etc) were led by the client needs who pay for it. Only recently, these processes have been opened to a broader community (firms and volunteers collaborating in the project within the gvSIG association). So, it makes sense that the beginnings had seen less activity (high phases of planing) and afterwards they got to agregate so many people in such a short period of time.
• But, in 2010 it suffered a sudden stop in development (only 233 changes to the codebase were made, while a pace of 4500 changes were made during previous years). This decreasing in activity is highly correlated to the number of developers involved. It’s hard to say why it happens: could it be due to the efforts were directed to gvSIG 2.0 development? could it be due to the reorganization in the project and the creation of gvSIG asociation? Well, few can we said at this respect with the data available, further research is required to determine that.

QGIS

The next one shows the same data (number of people writing and number of messages by month) for the developers mailinglists.

Is it something we could extrapolate from the data there?

Well, certainly not the user base. The data shyly introduce us the trends, not the real user base. The model we adopted to study the projects reflects just a part of the community -which is arguably the engine of project- but don’t take the data as the number of users for each project. For sure, each one of our favorite projects has more users than those participating in (these) mailinglists!

Anyway, here some food for thought:

• GRASS: it smoothly decreases in terms of number of messages as well as people writing, which happen within users and developers. The tendency is not clear though.
• gvSIG: the data shows a steadly increasing number of users participating in the mailinglists. On the other hand, although it is the project with more people suscribed to developer mailinglist, it shows the less activity of the three projects (in terms of # of messages in developer lists): few technical conversations seemed to happen through the mailinglists during that period.
• QGIS: according to the data, a clear growth exists in the community. In the period in study (3 years) the number of users and developers participating in mailinglists has been doubled!

• Route the user actions to the proper tool which will execute it.
• Manage the drawing of the layers.

This post covers the first of above tasks in an introductory way.

Flow of control

MapControl is a java component, which uses the idea of Chain of Responsibility to delegate work on others. Let’s see how it works in this case with a good old graphic:

1. MapControl listen MouseEvents through the MapToolListener. In response to an event, the MapToolListener will call the active tool (let’s say this class is named Behaviour).
2. The active Behaviour processes the info from the mouse, put together the contextual information needed (let’s call that an Event) and calls the next step in the chain (let’s call it the ToolListener).
3. Finally, the ToolListener will execute the actions needed to carry on the task an user have asked for.

public void mouseReleased(MouseEvent e) {

    try {

        if (currentMapTool != null)

        currentMapTool.mouseReleased(e);

    } catch (BehaviorException t) {

        throwException(t);

    }

}

public void mouseReleased(MouseEvent e) throws BehaviorException {

    if (e.getButton() == MouseEvent.BUTTON1 && m_FirstPoint!=null) {

        MoveEvent event = new MoveEvent(m_FirstPoint, e.getPoint(), e);

        listener.move(event);

    }

    m_FirstPoint = null;

}

public void move(MoveEvent event) {



    ViewPort vp = mapControl.getMapContext().getViewPort();

    Point2D from = vp.toMapPoint(event.getFrom());

    Point2D to = vp.toMapPoint(event.getTo());



    //build the new extent

    Rectangle2D.Double r = new Rectangle2D.Double();

    Rectangle2D extent = vp.getExtent();

    r.x = extent.getX() - (to.getX() - from.getX());

    r.y = extent.getY() - (to.getY() - from.getY());

    r.width = extent.getWidth();

    r.height = extent.getHeight();



    //update the ViewPort

    vp.setExtent(r);

}

Some useful resources about MapControl in gvSIG wiki:

• MapControl (spanish)
• Behaviour (spanish)
• ToolListeners (spanish)
• Events related to toollisteners (spanish)

• Mapcontrol
• Canonical events of MapControl:
• MeasureEvent
• MoveEvent
• PointEvent
• RectangleEvent

During a serie of posts begin with this, I’m going to show a quantitative analysis of communities from 3 reference projects in GIS arena: GRASS, gvSIG y QGIS. We selected those, as they are viewed as the more mature projects in desktop GIS, they are under OSGEO Fundation umbrella and show some differences on the actors who bootstrapped and manage today.

What we have done?

During the more than 25 years of free software movement, it has delighted us with the high capacity for fostering creation and innovation a community-based model has. Along last years, that model proved its viability in other areas too: content creation (wikipedia), cartographic data creation (openstreetmaps), translating books, etc. Yet, few is known on “how to bootstrap and grow a community”. The only thing we can do is observing what others have done and learn from their experience.

In order to contribute to the understanding on how a community-based project works I’ve work with Francisco Puga and other people from Cartolab to put together some of the public information the projects generate and make some sense from that. The actors in a community interact with each other, and, when that happen through internet, a trail is left (messages to mailinglists have author information and date, code version systems log information about the authors too, …). Basing our work on this available and public information -and standing on the shoulder on giants -i.e: reviewing a lot of research works similar to what we like to build- we have developed a quantitative analysis on the communities supporting GRASS, gvSIG and QGIS.

How did we make it?

The first step was to evaluate and gather all the public information a project, for what we like to do it in automated way. But, as we had to compare the 3 projects, the data had to be homogeneous: at least exists in both 3 and be in a comparable format. Taking these constraints into account (and the limited time we had for this!) we have collected information from 2 different systems:

• Code versions control systems: from every project, we cloned all information available in their repositories to a local git repo, in order to parse the log of changes. This allowed us to study all the history of projects, from the very begining to December 2010.
• Data was got from trunk: grass - gvsig - qgis.
• Mailinglists: by means of mailingliststats tool -built mainly by our friend Israel Herráiz, thanks bro!- we gather data from March 2008 to December 2010.
  • Development mailinglists: grass - gvsig - qgis
  • Users mailinglists: grass - gvsig en / gvsig es - qgis

Some disclaimers:

• Projects have a number of branches, plugins and so. We focused the study on the main product, what an user get when she downloads it. Further study on the plugins ecosystem is needed, and it will give us more fine-tuning information.
• Projects have a number of mailinglists more than we have studied (translators, steering committee, other local/regional mailinglists, etc), varying on each case. The analysis was focused on developers and users ones due to we think they are representative enough to mark the trend. We are not interested in giving an exact number (which may be impossible to measure!) but in drawing the long-term fluctuation of participation. Our intuition and past experiences, says that those mailinglists will follow a correlation of participation with the larger community surrounding the projects.
• In the particular case of gvSIG users mailinglists, we have studied spanish and english mailinglist jointly. It makes sense doing so as the spanish mailinglist still have the core of contributions from hispanoamerican countries and non-spanish people interacts through international mailinglist. It is like the project have two hearts.
• Unfortunately, quality of data have limited the period in study: the range is from March 2008 to December 2010. Prior to that, not all projects have information due to mailinglist migrations.

What is it useful for?

It’s possible to analyze a community from a variety of points of view. Our approach is a quantitative focus by means of a common model which agregate users depending on their level of participation:

• Leaders: those who build the product and make the decisions.
• Power users: those who adapt it to their needs and using it intensively.
• Casual users: those who using it for a concrete task.

This approach allow us to better understand the size of the community and how they interact, as it’s not the same the value provided by someone who in 6 months only sent 1 mail to a mailinglist than other person who spent that time sending more than 100 patches to the code.

With these constraints, we managed to built the following indicators:

• Adoption trend within users and developers: based on mailinglists data.
• Status: post published.
• Activity and manpower: based on code contributions (commits).
• Status: post published.
• Composition of the community: based on code contributions (commits).
• Status: still to be published.
• Generational analysis: based on code contributions (commits).
• Status: still to be published.

During next weeks, I will be publishing the results of the study, in order to help us to understand how different free software communities work, and what we can learn from that. Stay tunned!

Coda

The results shown here are borrowed from a paper I led jointly with Francisco Puga, Alberto Varela and Adrián Eirís from Cartolab, a GIS university research laboratory based on A Coruña. The results were shown on the V Jornadas de SIG Libre, Girona 2010. If you are fluent in spanish (reading or listening), you can benefit from these resources:

• (in spanish) The complete paper [PDF].
• (in spanish) The slides [PDF].
• (in spanish) Video explaining the highlights - not my best performance though

CADToolAdapter class in extCAD extension maintains a list of snappers and layers to snap to from the editing layer. When the mouse is moved, the snappers are recalculated following this algorithm (note that the code below is the core of the method, some other parts/casts and boilerplate code is missing):

ArrayList snappers = SnapConfigPage.getActivesSnappers();

ILayerEdited layerInEdition =

    CADExtension.getEditionManager().getActiveLayerEdited();

ArrayList layersToSnap = layerInEdition.getLayersToSnap();



for (FLyrVect layer : layersToSnap) {



    // Getting the set of geometries within the envelope

    // The envelope is calculated based on the tolerance the user wants

    SpatialCache cache = layer.getSpatialCache();

    List geometries = cache.query(envelope);



    // Updating the nearest point

    for (Feature geomToSnap : geometries){

        for (int i=0; i<snappers.size(); i++){

            Point2D pointToSnap = snappers[i].getSnapPoint(queryPoint,

                                                geomToSnap,

                                                tolerance,

                                                lastPointEntered);

        }

        double distance = pointToSnap.distance(queryPoint);

        if(minimunDistance > distance){

                minimunDistance = distance;

        }

    }

}

This algorithm is executed every time the user move the mouse and is very quick if you have few layers to snap to. But, as the number of layer to check increases, the editing process becomes very slow. Besides, as a comment of software design, after reviewing this part of code, I like the way the snappers fit in gvsig cad tools. If you want to add a new snapper, just need to implement ISnapperVectorial interface and make getSnapToPoint method to return the nearest point to the position of the mouse. So, designing your own snappers is very easy!

By the way, if you feel like replying how other GIS applications (QGIS, uDig, …) manage the snappers, I’d be more than happy to hear and learn that!

Some general impressions on the event

• It’s great to see codesprints are becoming usual in gvsig project. It’s still a new practice to be fully embraced for most members of the community but taking into account that the the first codesprint I proposed was almost 1 year ago, we have good rithm (this was the 4th codesprint!). It feels good to see such an amount of people and energy during the codesprint. It is encouraging and talks about their commitment to the project.
• This codesprint was the 2nd in number of participants after the one in Valencia! That confirms that Galicia matters That was the good news. The bad ones: I’ve missed developers from England (do you know London is directly connected to A Coruña by flight?), Portugal (don’t you know that we speak the same language?!) and more people, shops and gvSIG members & collaborators from Spain, though. We should try harder to bring people to codesprints.
• That kind of events are great to grow relationships and trust. Also they are great to communicate diffuse information and transfer knowledge: we had some good conversations with Joaquín del Cierro (gvSIG development manager) about the technological background of the project, how some decissions are made and the direction of the development theses days.

NavTable and NavTableForms: what we have done

In the hacking side, I’m pretty happy with the results. Most of the time, Jorge López and me paired together to resolve the priority things on NavTable and NavTableForms. Here the results:

NavTable

• Flavio Pompermaier had talked us about the differences between layer.getSource().getRecordset() and layer.getRecordset() methods. Roughly: the second returns the features directly from the source, which seems to reflect changes (schema modifications, add new registers, etc) in real time. I spent a time to reproduce the bugs he showed us but I couldn’t. The last work on listeners done by Javier Estévez and to be integrated in gvSIG 1.12 had solved it.
• Having some good conversations with Nacho regarding past conversations and improvements on NavTable UI. Some of the proposals will appear soon on your favorite mailinglist!
• Having some good conversations with Joaquín on metadata and filtering in gvSIG, which resulted on:
  • metadata: the better way to integrate metadata into gvSIG sources is by doing your own custom mechanism, as we already have (for example: to associate to the data domain-values or validation). There is no such a thing of a broad standard on the matter (although there are some attempts).
  • pre-filtering from datasource (definitionExpression in other GIS applications): I had asked that in the mailinglist (and even got a only-read solution). Lately, it appeared again in gvsig-international a thread talking about that. The short answer: it’s not possible to do it now.
• Refactoring to actions. I work on this during Friday morning. It was more difficult that I thought as our codebase is very coupled at some key part, which required an aditional work to do this. What I got is an initial prototype working (uploaded to a temporal branch on our repo) only with the moving actions (go-next, go-last, go-previous, go-first). Although still a prototype I think is very promising as a base to work on.

NavTableForms

• Landing into trunk, the big rewriting done in NavTableForms. Jorge and I spent thursday morning reading code, updating the example for it to work with the new code and with integrating issues. It was easier that I thought, and we had the example working in less than 3 hours. I even write down our mini-guide to migrate the example, as it could be interesting for other people.
• Add support for reading domain values from a text file. One of the news on NavTableForms is that it’s able to read domain values from a database (say postgress). Jorge worked to add support for textfiles in a similar way than we do for alias.
• Adding support for multiple validation rules. Jorge worked and integrated this very nice feature.
• A new validation rule: mandatory field. I commited this to repo. By the way, It’s nice to see how easy is to add a new rule.

Summing up: as you may see, they were two intense days! A lot of work done and the hacktivation energy at full again. Looking forward to next one!

A lo largo de una serie de post que inicio hoy voy a presentar un análisis de las comunidades de 3 proyectos de referencia en el mundo SIG: GRASS, gvSIG y QGIS. Durante el proceso de selección nos hemos quedado con estos 3 porque consideramos que son los más importantes y maduros SIG de escritorio, están además bajo el paraguas de la Fundación OSGEO y presentan diferencias en los actores que los gestionan.

¿Qué hemos hecho?

En los más de 25 años que tiene el movimiento del software libre, se ha demostrado la gran capacidad de creación que tiene un modelo centrado en la comunidad. Un modelo que, además, ha mostrado su viabilidad expandiéndose a otras áreas: creación de contenidos (wikipedia), creación de datos cartográficos (openstreetmaps), traducción de libros, etc. Pero si bien conocemos su potencia, poco sabemos sobre “cómo crear y gestionar una comunidad“. Lo único que podemos hacer es observar qué han hecho los demás y cómo les ha ido. Probar. Tratar de extrapolar heurísticos de la experiencia de otros.

Para contribruir al entendimiento de cómo funcionan las comunidades de software libre -Francisco Puga, otra gente del Cartolab y yo- hemos realizado un análisis de las comunidades en base a la información pública que generan. Los actores de una comunidad interactúan entre sí, y, cuando eso ocurre a través de internet, las interacciones dejan rastro:

• Listas de correo: los mensajes contienen la fecha, el autor, etc.
• Wiki: es posible obtener información sobre el autor, la fecha de creación, el número de ediciones de una página, etc.
• Sistemas de control de errores: información sobre quién y cuándo se reportó, si está resuelto o no, etc.
• Sistemas de control del código: podemos obtener la actividad sobre la aplicación basándonos en el número de cambios (commits), conocer quién los hizo, la fecha, etc.

Con la base de esta información pública disponible, lo que hemos hecho ha sido un estudio cuantitativo sobre las comunidades que rodean y sostienen a estos proyectos.

¿Cómo lo hemos hecho?

Gracias a la disponibilidad de ciertas herramientas que nos facilitaron el proceso de obtención de información, además de tener en cuenta la calidad de los datos para poder hacer comparativas entre los proyectos, lo que finalmente logramos hacer fue lo siguiente:

• Sistemas de control de código: hemos volcado toda la información disponible al sistema de control de versiones git para luego parsear su histórico. Esto nos ha permitido estudiar toda la historia de desarrollo de los proyectos hasta diciembre del 2010. Datos para grass – gvsig – qgis
• Listas de correo: hemos usado para ellos la herramienta mailingliststats -que construyó principalmente Israel Herráiz, thanks bro!- con datos desde marzo de 2008 hasta diciembre de 2010, en base a:
  • las listas de desarrollo del proyecto: grass – gvsig – qgis
  • las listas principales de usuarios: grass – gvsig en / gvsig es – qgis

Algunas aclaraciones sobre el estudio de las listas de correo:

• Los 3 proyectos tienen muchas más listas para diversos aspectos (traducciones, dirección del proyecto, listas locales, etc). Nos hemos centrado en éstas porque creemos que son suficientes para marcar la tendencia, que realmente es lo que nos interesa; no los números gordos que serían engañosos.
• En el caso de las listas de usuarios, para gvsig hemos estudiado además de la lista internacional, también la española. Ésta última es donde nació el proyecto y muestra todavía la actividad principal. No hacerlo introduciría sesgos.
• Por desgracia, la calidad de los datos nos ha limitado el período de estudio: hemos conseguido analizar desde Marzo de 2008 hasta diciembre del 2010.

¿Para qué nos vale?

El estudio de una comunidad tiene diferentes enfoques. El nuestro se basa en el modelo que divide a la comunidad en 3 niveles de participación e implicación:

• Leaders: aquellos que construyen el producto.
• Power users: aquellos que lo adaptan a sus necesidades y lo usan intensivamente.
• Casual users: aquellos que lo usan para una tarea concreta.

Esta aproximación facilita la comprensión de cómo funciona realmente la comunidad, ya que no es lo mismo la aportación de una persona a través de un único mensaje en una lista de correo a la de alguien que se ha pasado 6 meses creando la aplicación. Nos aporta además, información sobre la adopción de las herramientas así como patrones de participación y actividad entre los distintos actores.

Con este enfoque y metodología hemos conseguido realizar los siguientes indicadores:

• Tendencias de adopción entre usuarios: basado en las listas de correo.
• Tendencias de adopción entre desarrolladores: basado en las listas de correo.
• Actividad y fuerza de trabajo: basado en contribuciones de código (commits).
• Análisis de composición de la comunidad: basado en contribuciones de código.
• Análisis generacional: basado en contribuciones de código.

En las siguiente semanas iremos publicando los resultados del estudio, de cara a comprender mejor cómo funciona una comunidad de software libre. Stay tunned!