GeoTools

OSGeo

Monday, September 24, 2018

GeoTools 20.0 Released

The GeoTools team is pleased to announce the release of GeoTools 20.0:
This is the first release of the 20.x series, now marked as stable and deemed suitable for production systems, while 19.x switches to maintenance mode and 18.x moves to unsupported.

This release is also available from our Maven repository, and is made in conjunction with GeoServer 2.14.0.

This release includes a various major changes.

JTS upgraded to version 1.16

JTS has been upgraded to version 1.16. This marks a significant change in package naming, the library has switched from "com.vividsolutions" to "org.locationtech"  packages and all GeoTools code has been updated to follow suit.

JTS classes are widely used in GeoTools, as a result the client code using GeoTools will also have to be modified to follow suite. Thankfully the changes amount to a search and replace and can be easily automated, please check the upgrade instructions available in the GeoTools site.

Units library migrated to JSR-363

GeoTools was using an aging and un-maintained units library and this development round moved it to a modern and maintained set of packages compatible with Java 10. Like the JTS upgrade this might require the client code to be updated, both in terms of packages and in terms of classes and methods being used.

We have provided a Units utility class, providing an easy way to match unit defined at runtime with the the correct constant.
Unit deg = Units.autoCorrect(SI.RADIAN.multiply(0.0174532925199433));
Please check the upgrade instructions available in the GeoTools site for more details.

Thanks to César Martínez Izquierdo and the participants of the OSGeo Code sprint for this work.

Various other assorted dependencies upgrades

The release upgrades many dependencies to newer versions, in particular:
  • Batik upgraded to version 1.10
  • Commons-lang to version 3.7
  • Commons-io to version 2.6
  • Mysql JDBC driver to 5.1.46
  • HSQLDB to 2.4.1
  • SQLite-JDBC to version 2.23.1
  • GeographicLib upgrades to version 1.49
These upgrades should not cause major incompatibilities, but look for for potential API changes.

Channel selection name allow expressions

GeoTools 20.x allows expressions to be used in SourceChannelName SLD elements, and in their code counterpart, thus allowing dynamic channel selection. This is welcomed news for anyone building applications that display multispectral or hyperspectral data, thus avoiding the need to build many different styles for the various interesting false color combinations.

Here is an SLD example:

<RasterSymbolizer>
  <ChannelSelection>
    <RedChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B1</ogc:Literal>
             <ogc:Literal>2</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </RedChannel>
    <GreenChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B2</ogc:Literal>
             <ogc:Literal>5</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </GreenChannel>
    <BlueChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B3</ogc:Literal>
             <ogc:Literal>7</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </BlueChannel>
  </ChannelSelection>
<RasterSymbolizer>

Map algebra

This release adds support for an efficient map algebra package knows as Jiffle. Jiffle has been the work of a former GeoTools contributor, Michael Bedwards, that has been salvaged, upgraded to support Java 8, and integrated in jai-ext. From the there support has been added into the GeoTools gt-process-raster module, to be used either directly or as a rendering transformation.

The following SLD style calls onto Jiffle to perform a NDVI calculation on top of Sentinel 2 data:

<?xml version="1.0" encoding="UTF-8"?>
<StyledLayerDescriptor xmlns="http://www.opengis.net/sld" xmlns:ogc="http://www.opengis.net/ogc" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/sld
http://schemas.opengis.net/sld/1.0.0/StyledLayerDescriptor.xsd" version="1.0.0">
  <NamedLayer>
    <Name>Sentinel2 NDVI</Name>
    <UserStyle>
      <Title>NDVI</Title>
      <FeatureTypeStyle>
        <Transformation>
          <ogc:Function name="ras:Jiffle">
            <ogc:Function name="parameter">
              <ogc:Literal>coverage</ogc:Literal>
            </ogc:Function>
            <ogc:Function name="parameter">
              <ogc:Literal>script</ogc:Literal>
              <ogc:Literal>
                nir = src[7];
                vir = src[3];
                dest = (nir - vir) / (nir + vir);
              </ogc:Literal>
            </ogc:Function>
          </ogc:Function>
        </Transformation>
        <Rule>
          <RasterSymbolizer>
            <Opacity>1.0</Opacity>
            <ColorMap>
              <ColorMapEntry color="#000000" quantity="-1"/>
              <ColorMapEntry color="#0000ff" quantity="-0.75"/>
              <ColorMapEntry color="#ff00ff" quantity="-0.25"/>
              <ColorMapEntry color="#ff0000" quantity="0"/>
              <ColorMapEntry color="#ffff00" quantity="0.5"/>
              <ColorMapEntry color="#00ff00" quantity="1"/>
            </ColorMap>
          </RasterSymbolizer>
        </Rule>
      </FeatureTypeStyle>
    </UserStyle>
  </NamedLayer>
</StyledLayerDescriptor>
The performance is good enough for interactive display, and the result looks as follows:

PostGIS store improvements

The PostGIS datastore has been for years the only one that could encode a few filter functions used in filters down into native SQL, but it required a datastore creation flag to be enabled.
Starting with this release it will do so by default.


The functions supported for SQL encoding by the store are:
  • String functions: strConcat, strEndsWith, strStartsWith, strEqualsIgnoreCase, strIndexOf, strLength, strToLowerCase, strToUpperCase, strReplace, strSubstring, strSubstringStart, strTrim, strTrim2
  • Math functions: abs, abs_2, abs_3, abs_4, ceil, floor
  • Date functions: dateDifference
Along with improvements in the handling of GroupByVisitor, this makes for efficient histogram computation directly in the database, while retaining the ability to share the same code with other stores that do not have the same optimizations.

This release adds support for "array" data type in the store, with full reading and writing support, as well as native filtering (with index support, where feasible).

Finally, it's now possible to read geometries with measures from PostGIS, also thanks to JTS improved CoordinateSequence API, and encode the results in GML (the encoding is subject to a configuration flag, as GML does not officially support M encoding). The work will continue in the next few month in order to cover more formats.

Image mosaic improvements

The image mosaic module never sleeps, in this release we see the following improvements:
  • Support for remote images (e.g. S3 or Minio). In order to leverage this the mosaic index will have to be created up-front (manually, or with some home grown tools)
  • A new "virtual native resolution" read parameter allows the mosaic to compose outputs respecting a native resolution other than its native one (useful in case you want to give selective resolution access to different users)
  • Supporting multiple WKBs footprint for pixel precise overviews masking
  • A new read mask parameter allows to cut the image to a given geometry (again, helps in providing different selective views to different users)
  • Speed up NetCDF mosaics by allowing usage of stores coming from a repository, instead of re-creating them every time a NetCDF file is needed (to be setup in the auxiliary store config file, while the repository instance needs to be passed as a creation hint).
  • The mosaic now works against images without a proper CRS, setting them into the "CRS not found" wildcard CRS, aka "EPSG:404000"

App-schema improvements

The app-schema module got significant performance and functionality improvements since 19.x series, in particular:
  • Better delegation of spatial filters on nested properties to native database
  • Improved support for multiple nested attribute matches in filters
  • It is now possible to use Apache SOLR as a data source for app-schema
  • The configuration files can be hosted on a remote HTTP server

Modules moving up and down

This release sees a number of modules moving down to unsupported due to lack of maitainers, unsupported module being removed, while other modules graduate to supported status. In particular:
  • The unsupported image-collection, sfs, efeature, caching, feature-aggregate and geotiff-new module have been removed from the code base (you can still find them in older releases and version control history)
  • The OGR and GTOPO30 modules have been downgraded to unsupported status due to lack of maintainership
  • The MongoDB module moves up to supported status

Building on Windows

The GeoTools project builds again on Windows (with tests) and all new pull requests are verified using AppVeyor.

GeoTools is looking for Windows developers to join and help keep the build passing, as well as locating and squashing a few intermittent failures we're still experiencing.

Assorted improvements

Other highlights from our issue tracker release-notes:
  • Some speedups in in memory Expression evaluation, in particular attribute value extraction in simple features and equality tests
  • Support for "none" values in GeoCSS (to allow turning off symbolization/properties in override rules)
There is also a large set of bug fixes. For more information see the release notes: (20.0 | 20-RC).

Tuesday, September 18, 2018

GeoTools 20 Release Candidate Available

The GeoTools team is pleased to announce the release of GeoTools 20-RC:
This release candidate is also available from our Maven repository, and is made in conjunction with GeoServer 2.14-RC.

This release includes a various major changes.

Please Test this Release Candidate

A release candidate is your chance to both try out new features and contribute to the project with valuable feedback when we need it most.

Please test this release and let us know of any regression before we release GeoTools 20.0 final, in the second half of September.

JTS upgraded to version 1.16

JTS has been upgraded to version 1.16. This marks a significant change in package naming, the library has switched from "com.vividsolutions" to "org.locationtech"  packages and all GeoTools code has been updated to follow suit.

JTS classes are widely used in GeoTools, as a result the client code using GeoTools will also have to be modified to follow suite. Thankfully the changes amount to a search and replace and can be easily automated, please check the upgrade instructions available in the GeoTools site.

Units library migrated to JSR-363

GeoTools was using an aging and un-maintained units library and this development round moved it to a modern and maintained set of packages compatible with Java 10. Like the JTS upgrade this might require the client code to be updated, both in terms of packages and in terms of classes and methods being used.

We have provided a Units utility class, providing an easy way to match unit defined at runtime with the the correct constant.
Unit deg = Units.autoCorrect(SI.RADIAN.multiply(0.0174532925199433));
Please check the upgrade instructions available in the GeoTools site for more details.

Thanks to César Martínez Izquierdo and the participants of the OSGeo Code sprint for this work.

Various other assorted dependencies upgrades

The release upgrades many dependencies to newer versions, in particular:
  • Batik upgraded to version 1.10
  • Commons-lang to version 3.7
  • Commons-io to version 2.6
  • Mysql JDBC driver to 5.1.46
  • HSQLDB to 2.4.1
  • SQLite-JDBC to version 3.23.1
  • GeographicLib upgrades to version 1.49
These upgrades should not cause major incompatibilities, but look for for potential API changes.

Channel selection name allow expressions

GeoTools 20.x allows expressions to be used in SourceChannelName SLD elements, and in their code counterpart, thus allowing dynamic channel selection. This is welcomed news for anyone building applications that display multispectral or hyperspectral data, thus avoiding the need to build many different styles for the various interesting false color combinations.

Here is an SLD example:

<RasterSymbolizer>
  <ChannelSelection>
    <RedChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B1</ogc:Literal>
             <ogc:Literal>2</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </RedChannel>
    <GreenChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B2</ogc:Literal>
             <ogc:Literal>5</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </GreenChannel>
    <BlueChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B3</ogc:Literal>
             <ogc:Literal>7</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </BlueChannel>
  </ChannelSelection>
<RasterSymbolizer>

Map algebra

This release adds support for an efficient map algebra package knows as Jiffle. Jiffle has been the work of a former GeoTools contributor, Michael Bedwards, that has been salvaged, upgraded to support Java 8, and integrated in jai-ext. From the there support has been added into the GeoTools gt-process-raster module, to be used either directly or as a rendering transformation.

The following SLD style calls onto Jiffle to perform a NDVI calculation on top of Sentinel 2 data:

<?xml version="1.0" encoding="UTF-8"?>
<StyledLayerDescriptor xmlns="http://www.opengis.net/sld" xmlns:ogc="http://www.opengis.net/ogc" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/sld
http://schemas.opengis.net/sld/1.0.0/StyledLayerDescriptor.xsd" version="1.0.0">
  <NamedLayer>
    <Name>Sentinel2 NDVI</Name>
    <UserStyle>
      <Title>NDVI</Title>
      <FeatureTypeStyle>
        <Transformation>
          <ogc:Function name="ras:Jiffle">
            <ogc:Function name="parameter">
              <ogc:Literal>coverage</ogc:Literal>
            </ogc:Function>
            <ogc:Function name="parameter">
              <ogc:Literal>script</ogc:Literal>
              <ogc:Literal>
                nir = src[7];
                vir = src[3];
                dest = (nir - vir) / (nir + vir);
              </ogc:Literal>
            </ogc:Function>
          </ogc:Function>
        </Transformation>
        <Rule>
          <RasterSymbolizer>
            <Opacity>1.0</Opacity>
            <ColorMap>
              <ColorMapEntry color="#000000" quantity="-1"/>
              <ColorMapEntry color="#0000ff" quantity="-0.75"/>
              <ColorMapEntry color="#ff00ff" quantity="-0.25"/>
              <ColorMapEntry color="#ff0000" quantity="0"/>
              <ColorMapEntry color="#ffff00" quantity="0.5"/>
              <ColorMapEntry color="#00ff00" quantity="1"/>
            </ColorMap>
          </RasterSymbolizer>
        </Rule>
      </FeatureTypeStyle>
    </UserStyle>
  </NamedLayer>
</StyledLayerDescriptor>
The performance is good enough for interactive display, and the result looks as follows:

PostGIS store improvements

The PostGIS datastore has been for years the only one that could encode a few filter functions used in filters down into native SQL, but it required a datastore creation flag to be enabled.
Starting with this release it will do so by default.


The functions supported for SQL encoding by the store are:
  • String functions: strConcat, strEndsWith, strStartsWith, strEqualsIgnoreCase, strIndexOf, strLength, strToLowerCase, strToUpperCase, strReplace, strSubstring, strSubstringStart, strTrim, strTrim2
  • Math functions: abs, abs_2, abs_3, abs_4, ceil, floor
  • Date functions: dateDifference
Along with improvements in the handling of GroupByVisitor, this makes for efficient histogram computation directly in the database, while retaining the ability to share the same code with other stores that do not have the same optimizations.

This release adds support for "array" data type in the store, with full reading and writing support, as well as native filtering (with index support, where feasible).

Finally, it's now possible to read geometries with measures from PostGIS, also thanks to JTS improved CoordinateSequence API, and encode the results in GML (the encoding is subject to a configuration flag, as GML does not officially support M encoding). The work will continue in the next few month in order to cover more formats.

Image mosaic improvements

The image mosaic module never sleeps, in this release we see the following improvements:
  • Support for remote images (e.g. S3 or Minio). In order to leverage this the mosaic index will have to be created up-front (manually, or with some home grown tools)
  • A new "virtual native resolution" read parameter allows the mosaic to compose outputs respecting a native resolution other than its native one (useful in case you want to give selective resolution access to different users)
  • Supporting multiple WKBs footprint for pixel precise overviews masking
  • A new read mask parameter allows to cut the image to a given geometry (again, helps in providing different selective views to different users)
  • Speed up NetCDF mosaics by allowing usage of stores coming from a repository, instead of re-creating them every time a NetCDF file is needed (to be setup in the auxiliary store config file, while the repository instance needs to be passed as a creation hint).
  • The mosaic now works against images without a proper CRS, setting them into the "CRS not found" wildcard CRS, aka "EPSG:404000"

App-schema improvements

The app-schema module got significant performance and functionality improvements since 19.x series, in particular:
  • Better delegation of spatial filters on nested properties to native database
  • Improved support for multiple nested attribute matches in filters
  • It is now possible to use Apache SOLR as a data source for app-schema
  • The configuration files can be hosted on a remote HTTP server

Modules moving up and down

This release sees a number of modules moving down to unsupported due to lack of maitainers, unsupported module being removed, while other modules graduate to supported status. In particular:
  • The unsupported image-collection, sfs, efeature, caching, feature-aggregate and geotiff-new module have been removed from the code base (you can still find them in older releases and version control history)
  • The OGR and GTOPO30 modules have been downgraded to unsupported status due to lack of maintainership
  • The MongoDB module moves up to supported status

Building on Windows

The GeoTools project builds again on Windows (with tests) and all new pull requests are verified using AppVeyor.

GeoTools is looking for Windows developers to join and help keep the build passing, as well as locating and squashing a few intermittent failures we're still experiencing.

Assorted improvements

Other highlights from our issue tracker release-notes:
  • Some speedups in in memory Expression evaluation, in particular attribute value extraction in simple features and equality tests
  • Support for "none" values in GeoCSS (to allow turning off symbolization/properties in override rules)
There is also a large set of bug fixes. For more information see the 20-RC release notes.