FOSS4GNA Code Sprint Replacing Vecmath

As Torben indicated on the geoserver blog we got together for a one day sprint after the foss4gna conference. Torben stole my picture for that post so I will have to stick to content ...

Our topic of choice ... replacing vecmath. The use of vecmath library has been a long standing "technical debt" for GeoTools. How long standing? The issue tracker number is GEOT-22.

So what is the problem with vecmath

The vecmath.jar is used by gt-referencing for coordinate system transformations. We only uses a couple classes form the library: primarily to implement GeneralMatrix for use in MathTransforms.

GeneralMatrix extending GMatrix

There is one small problem with this idea - vecmath is not open source! Technically vecmath is distributed as part of Java 3D (an extensions to the Java Virtual Machine). As an extension to Java it was distributed under the same licenses (Sun Binary License and a Research License) as Java.

With the GeoGig project going through LocationTech incubation we have a couple ways to use jars:

• prerequisite: open source jars required to run
• works with: optional jars that extend the functionality if present. These may be proprietary like an oracle database driver.

Although the vecmath license is fine for distribution it does not meet the strictly open source policies required for the GeoGig project. In this case we want GeoTools to include matrix math and needed to shop around for a replacement.

The use of vecmath (as a non open source dependency) also causes trouble for Rich Fecher's proposal to publish GeoTools on Maven Central. 

Enter EJML

With a technical dept page capturing research, some email discussion and a great lunchtime conversation at foss4gna with Eric Engle (the overlap with EclipseCon was good for something) we settled on the recommended Efficient Java Matrix Library (EJML).

GeneralMatrix delegating to DenceMatrix64F

The strategy here is delegate to the DenseMatrix64F implementation provided by EJML, and implement the methods we expect from XMatrix in terms of this new implementation.

The EJML library has the similar arrangement with SimpleMatrix wrapping a DenseMatrix64F an API friendly to casual developers. We were able to use SimpleMatrix as a guide, saving a lot of time.

How to help

While the code sprint was a success in proving the approach, there is a bit of work to go:

• Tyler (from GeoGig) is working on removing the dependency on vecmath (there are a few other Exceptions and data structures in our API that need to be removed).
• Jim (from GeoMesa) wants to write up more test cases to check for regressions between vecmath and EJML
• Although Rich Fecher (from GeoWave) was unavailable to take part in the code sprint - his inspiration to work on this now means we will be hitting him up for a review when the work is complete. Thanks Rich!
• And there is always the question of performance ... will it be faster!

To help take part, or review our work see this branch in github:

• https://github.com/geotools/geotools/tree/matrix

Thanks to Boundless victoria staff, Jim and Andrea for really getting behind this work. This kind of up keep keeps the community ticking along and helps the library be used in more places.

I would also like to thank the new crop of projects using GeoTools for taking part and contributing upstream. It is important to keep our community diverse and your participation is both welcomed and appreciated.

CodeHaus Migration Schedule

As per earlier blog post CodeHaus is shutting down and the GeoTools project is taking steps to migrate our issue tracker and wiki to a new home.

First up I need to thank the Open Source Geospatial Foundation for responding quickly in a productive fashion. The board and Alex Mandel were in position to respond quickly and hire a contractor to work with the system admin committee to capture this content while it is still available.

I should also thank Boundless for providing me time coordinate CodeHaus migration and Andrea for arranging cloud hosting.

Updates:

• Update April 7th: GeoAPI project attachments migrated (using this to estimate time remaining)
• Update April 3rd: Issue tracker signup open to all (no need to email project leads).
• Update April 2nd: Mauro Bartolomeo created GEOT-5074 in the new issue tracker
• Update March 28th: Placeholder tickets created, contents and attachments to follow

Confluence Migration

Is scheduled for ... March 26th! I have taken a copy of the CodeHaus wiki and will be migrating proposals and project history. A html dump of the wiki is published at old.geotools.org so we have a record.

The new wiki is available here: https://github.com/geotools/geotools/wiki

GitHub Wiki

Jira Migration

Jira migration will start on 00:00 UTC Saturday March 28th.

On Saturday all issues will start migrating to their new home (and CodeHaus issue creation will be disabled). If you wish to lend a hand testing please drop by the #osgeo IRC channel on Saturday. Harrison Grundy will be coordinating the proceedings.

We have set up a new JIRA available at osgeo-org.atlassian.net for the migration. You can sign up directly (although we ask you to consider keeping the same user name).

OSGeo Jira

As shown above a few friendly CodeHaus refugees have also been sheltered for the storm (uDig and GeoAPI).

GeoTools 13.0 Released

GeoTools 13.0 Released

The GeoTools community is pleased to announce the availability of GeoTools 13.0, the first stable release of the GeoTools 13 series.

Download GeoTools 13:

• geotools-13.0-bin.zip
• geotools-13.0-doc.zip
• geotools-13.0-userguide.zip
• geotools-13.0-project.zip

This release is also available from our maven repository, and is made in conjunction with GeoWebCache 1.7.0 and GeoServer 2.7.0.

New Features and Improvements

The following new features and improvements are available in this release:

Data Stores

• GeoPackage support is now compatible with QGIS and OGR.
• A new gt-solr data store is available for working with with Apache Solr. Thanks to Andrea Aime and Justin Deoliveira for this work.
• CSVDataStore has been improved with write access and the ability to work with different kinds of CSV files (including lat/lon and WKT). Thanks to Travis Brundage for this work.
• AbstractDataStore is now deprecated. Please make plans to migrate to ContentDataStore. There is an extensive ContentDataStore tutorial to help with your migration.
• PropertyDataStore has now been ported to use ContentDataStore, and a wide range of issues (readers, events, transactions) have been resolved for all DataStore implementations. Thanks to Torben Barsballe for this extensive QA work.

CSS Module

• The gt-css module is a brand-new implementation of CSS-style support, now written in Java. Thanks to Andrea Aime for bringing this exciting approach to styling within easy reach of Java developers. Here is a quick code example:

Stylesheet ss = CssParser.parse(css);
CssTranslator translator = new CssTranslator();
Style style - translator.translate(ss);

Rendering Engine

• Color-blending has been introduced as both a FeatureTypeStyle and Symbolizer vendorExtension, allowing for a range of special effects. Thanks to Cleveland Metro Parks for this improvement.
• A FeatureTypeStyle vendorExtension named "firstMatch". This vendorExtension will grab the first rule you specify. This can in many situations help you avoid more complicated logic for style rules.
  

  <sld:VendorOption name="ruleEvaluation">first</sld:VendorOption>

• Anchor points are now supported. This is done in SLD with the AnchorPoint tag:
  

  <LabelPlacement>
    <PointPlacement>
      <AnchorPoint>
        <AnchorPointX>
          <PropertyName>X</PropertyName>
        </AnchorPointX>
        <AnchorPointY>
          <PropertyName>Y</PropertyName>
        </AnchorPointY>
      </AnchorPoint>
    </PointPlacement>
  </LabelPlacement>

WFS Client

The gt-wfs client has improved:

• Better compatibility with MapServer.
• Extensive work has been done to support WFS 2.0 transactions. Thanks to Niels Charlier for this work.
• The gt-wfs client now supports WFS 2.0 stored queries. Thanks to Sampo Savolainen for this work.

And more
Documentation continues to improve, now with a complete function list. For a full list of improvements and changes in GeoTools 13, please see the full release notes.

Thanks to Jody Garnett and Travis Brundage (from Boundless) for making this release happen.

CodeHaus Migration

It with heavy hearts (and a small bit of panic) that we bid adieu to CodeHaus. After many years of service CodeHaus is shutting down. A bit of last moment sponsorship from sonarqube is holding the doors open giving us a chance to migrate in an orderly fashion.

This blog post is a public notice that the GeoTools project will require assistance during this migration due to the short time frames involved.

GeoTools will lock down changes to the issue tracker after the scheduled March 18th release of GeoTools 13.0.

Migration Plan

In this weeks community meeting we have set on a migration strategy.

• Migrate GeoTools issue tracker to a new instance Jira (sorting out host options see below). This is primarily so we can manage large attachments (such as test data) that make our issue tracker valuable. As a fallback plan we will consider migration to github issues.
• Migrate GeoTools wiki to github wiki with a focus on proposals and project history.

Similar arrangements are being made for our friends with the GeoServer (OSGeo), uDig (LocationTech) and GeoAPI (OGC) projects. If you are in position to assist these projects please contact the appropriate development list as they could use a hand.

Due to a technical limitation (the use of a single sign on system) we cannot just take a backup and restore on a new system. This leaves us with two options: Use REST API and a script to migrate to a new system, or assemble a work party to transfer issues via CSV export (and grab the attachments one at a time).

Thus far we have contacted:

• The OSGeo board has offered their support in todays meeting for both setting up hosting and requesting a proposal for data transfer via REST API.
• The OSGeo system administration committee are looking into options for hosting.
  Thanks to both Harrison Grundy and Alex Mandel for helping refine migration requirements on IRC.
• Both GeoSolutions and Boundless have offered assistance
• We have reached out to Atlassian about cloud hosting

Thank you to the above groups for their prompt assistance.

About CodeHaus

CodeHaus has been an amazing resource for our open source community, offering friendly hands on support in the face of the growing pains experienced with SourceForge. We should also thank Atlassian for making their software (Jira and Confluence) available to open source projects on CodeHaus.