Monday, December 31, 2018

GeoTools 21-M0 Milestone Released

GeoTools project is closing out 2018 with a GeoTools 21-M0 milestone release featuring Java 11 compatibility:
This milestone release is a preview of GeoTools 21 giving everyone a chance to help test Java 11 compatibility.

This release works with both Java 8 and Java 9, downstream projects are advised that some imports will need to be fixed when upgrading as classes were repackaged on the way to Java 11 compatibility.

We have worked very hard for this release to work with both Java 8 and Java 11. By making a milestone release available at this time we are offering GeoTools users and downstream projects the option of migrating to Java 11 when they are ready.

Java 2018 Code Sprint

This release completes our participation in the OSGeo Java 2018 Code Sprint, and is made in conjunction with GeoServer 2.15-M0 and GeoWebCache 1.15-M0.

Thanks to individuals and organizations which took part in the code sprint: Boundless, GeoSolutions, GeoCatAstun Technology, and CCRi. Additional thanks to our sponsors whose financial support helps make these activities possible: silver sponsor Gaia3D, bronze sponsors atol, osgeo:uk, and Astun Technology.

Library Restructure and Upgrading

As part of this work many GeoTools dependencies were updated, and the library was repackaged for use as Java 11 modules. The most significant change is the removal of the gt-api module with its classes being distributed across the other modules.

A key technical requirement for Java 11 compatibility is avoiding having two modules making use of the same package. To avoid this "split-package" error some classes have been moved to different packages.

For additional details see the upgrade instructions in our user guide.

About Java 11 Compatibility

The GeoTools library is now available for use on the module path, see our Java 11 Quickstart for an example:

module org.geotools.tutorial.quickstart {
   requires java.desktop;
   requires org.geotools.main;
   requires org.geotools.shapefile;
   requires org.geotools.swing;
   requires org.geotools.render;
}

Java Roadmap Compatility

We are pleased to announce that the GeoTools library can now be built with Java 8 or Java 11! Due to a technical limitation with the core java class libraries our releases will be built with Java 8 for the widest compatibility.

Our initial concern with Oracle JDK 8 reaching its end-of-life in January 2019 has been alleviated by recent industry developments. Extensive Java 8 support options are now available, with RedHat making a public commitment to contribute fixes to the Java 8 codebase, and a range of organizations committed to making Java 8 builds available on a range of platforms.

Java 8 ProviderLicenseLinuxmacOSSolarisWindowsFree Updates
Oracle JDKBinary Code Licenexxxx2019 January
Oracle OpenJDKGPLxreference only
Oracle OpenJDKBinary Code Licenexxreference only
RedHat OpenJDKGPLx2023 June
Adopt OpenJDKGPLxxx2023 September

Oracle has contributed a wide range of technologies to the OpenJDK project which now leads Java 11 roadmap going forward. 

Java 11 ProviderLicenseLinuxmacOSSolarisWindowsFree Updates
Oracle JDKBinary Code Licenexxxx2019 March
Oracle OpenJDKGPLxxx2019 March
RedHat OpenJDKGPLx2024 October
Adopt OpenJDKGPLxxx2022 September

About GeoTools 21

GeoTools 21 series:

Sunday, December 23, 2018

GeoTools 19.4 released

The GeoTools team is pleased to announce the release of GeoTools 19.4:
This release is the last community sponsored maintenance release for the 19.x series and as such users and downstream projects should consider moving to the 20.x series.

This release is made in conjunction with GeoServer 2.13.4 and GeoWebCache 1.13.4.

Highlights from our issue tracker release-notes:

Tasks and Improvements:
  • Relaxed FilterDOMParser behavior vs like expressions, allowing generic expressions instead of PropertyName
Bug Fixes:
  • Failing bound computation in Oracle store fixed
  • GeoPackage raster to use JAI tile cache in mosaic operations (with significant speedups if there is a reprojection operation following the read), and properly exposing number of overviews and resolutions available (used for example by GeoServer WCS download limits)
  • Better support for filter reprojection on complex feature stores
  • Corrected envelope reprojection error from mercator to other CRS when working over small areas (e.g., tens of meters wide)
See release notes for this and previous 19.x releases for more details (19.3 19.2 19.1 19.0 19-RC1 19-beta).

Tuesday, November 20, 2018

GeoTools 20.1 Released

The GeoTools team is happy to announce the release of GeoTools 20.1:
This release is a stable release and is recommend for new development and production systems.

This release is made in conjunction with GeoServer 2.14.1.

Improvements

  • GML respects formatting options for optimized or 
  • ImageWorker allows omitting alpha band when converting from index color model to component color model.
  • Image mosaic property collector to record current time

Fixes

  • GeoPackage performance improved when working with large rasters
  • GML 3.1 fix for xml:id encoding
  • Fix for oracle select bounds query
  • Use of tile cache for GeoPackage mosaic operation
  • Interpolation ignore alpha band when scaling images
  • For more details see release notes (20.1 | 20.0 | 20-RC)

About GeoTools 20 Series

  • JTS Upgrade to version 1.16 and Migrate to JSR-363, see update instructions for details.
  • Channel selection name allows expression
  • Jiffle map algebra
  • PostGIS store improvements optimized to encode additional functions directly into SQL, support for array data types, and geometries with measure.


Thursday, October 18, 2018

GeoTools 19.3 released

The GeoTools team is pleased to announce the release of GeoTools 19.3:
This release is the current stable release and as such users and downstream projects should consider moving from older releases to this one.

This release is made in conjunction with GeoServer 2.13.3 and GeoWebCache 1.13.3.

GeoTools is proud to be an OSGeo project, and would like to thank Jürgen Fische and the OSGeo System Administration Committee for their infrastructure support this release.

Highlights from our issue tracker release-notes:

Tasks and Improvements:
  • Added a "NativeFilter" concept (to be used in particular stores offering more filtering options than the OGC inspired filters available in GeoTools)
  • App-Schema got a couple of interesting changes, extend it to make it possible to use an HTTP URL for the mapping file location, pass spatial filters on nested properties to the relational database
  • Allow the usage of a place holder for the :where: clause created by GeoServer in virtual tables
Bug Fixes:
  • Avoid interpolating alpha band when using bilinear or bicubic interpolation (broke masking)
  • Various image mosaic fixes and improvements, like better nodata management in some odd cases, better performance when the mosaic is working off a store provided by a repository (e.g. when referencing a GeoServer store by name), allow to mosaic togheter a mix of gray-alpha and RGBA images
  • CSS did not allow to support mark offset/anchor based on expressions while SLD did
  • The "gml:id" attribute was wrongly present in curved geometry GML elements in GML 3, while it's only required in version 3.2
See release notes for this and previous 19.x releases for more details (19.3 19.2 19.1 19.0 19-RC1 19-beta).

Monday, September 24, 2018

GeoTools 20.0 Released

The GeoTools team is pleased to announce the release of GeoTools 20.0:
This is the first release of the 20.x series, now marked as stable and deemed suitable for production systems, while 19.x switches to maintenance mode and 18.x moves to unsupported.

This release is also available from our Maven repository, and is made in conjunction with GeoServer 2.14.0.

This release includes a various major changes.

JTS upgraded to version 1.16

JTS has been upgraded to version 1.16. This marks a significant change in package naming, the library has switched from "com.vividsolutions" to "org.locationtech"  packages and all GeoTools code has been updated to follow suit.

JTS classes are widely used in GeoTools, as a result the client code using GeoTools will also have to be modified to follow suite. Thankfully the changes amount to a search and replace and can be easily automated, please check the upgrade instructions available in the GeoTools site.

Units library migrated to JSR-363

GeoTools was using an aging and un-maintained units library and this development round moved it to a modern and maintained set of packages compatible with Java 10. Like the JTS upgrade this might require the client code to be updated, both in terms of packages and in terms of classes and methods being used.

We have provided a Units utility class, providing an easy way to match unit defined at runtime with the the correct constant.
Unit deg = Units.autoCorrect(SI.RADIAN.multiply(0.0174532925199433));
Please check the upgrade instructions available in the GeoTools site for more details.

Thanks to César Martínez Izquierdo and the participants of the OSGeo Code sprint for this work.

Various other assorted dependencies upgrades

The release upgrades many dependencies to newer versions, in particular:
  • Batik upgraded to version 1.10
  • Commons-lang to version 3.7
  • Commons-io to version 2.6
  • Mysql JDBC driver to 5.1.46
  • HSQLDB to 2.4.1
  • SQLite-JDBC to version 2.23.1
  • GeographicLib upgrades to version 1.49
These upgrades should not cause major incompatibilities, but look for for potential API changes.

Channel selection name allow expressions

GeoTools 20.x allows expressions to be used in SourceChannelName SLD elements, and in their code counterpart, thus allowing dynamic channel selection. This is welcomed news for anyone building applications that display multispectral or hyperspectral data, thus avoiding the need to build many different styles for the various interesting false color combinations.

Here is an SLD example:

<RasterSymbolizer>
  <ChannelSelection>
    <RedChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B1</ogc:Literal>
             <ogc:Literal>2</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </RedChannel>
    <GreenChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B2</ogc:Literal>
             <ogc:Literal>5</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </GreenChannel>
    <BlueChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B3</ogc:Literal>
             <ogc:Literal>7</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </BlueChannel>
  </ChannelSelection>
<RasterSymbolizer>

Map algebra

This release adds support for an efficient map algebra package knows as Jiffle. Jiffle has been the work of a former GeoTools contributor, Michael Bedwards, that has been salvaged, upgraded to support Java 8, and integrated in jai-ext. From the there support has been added into the GeoTools gt-process-raster module, to be used either directly or as a rendering transformation.

The following SLD style calls onto Jiffle to perform a NDVI calculation on top of Sentinel 2 data:

<?xml version="1.0" encoding="UTF-8"?>
<StyledLayerDescriptor xmlns="http://www.opengis.net/sld" xmlns:ogc="http://www.opengis.net/ogc" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/sld
http://schemas.opengis.net/sld/1.0.0/StyledLayerDescriptor.xsd" version="1.0.0">
  <NamedLayer>
    <Name>Sentinel2 NDVI</Name>
    <UserStyle>
      <Title>NDVI</Title>
      <FeatureTypeStyle>
        <Transformation>
          <ogc:Function name="ras:Jiffle">
            <ogc:Function name="parameter">
              <ogc:Literal>coverage</ogc:Literal>
            </ogc:Function>
            <ogc:Function name="parameter">
              <ogc:Literal>script</ogc:Literal>
              <ogc:Literal>
                nir = src[7];
                vir = src[3];
                dest = (nir - vir) / (nir + vir);
              </ogc:Literal>
            </ogc:Function>
          </ogc:Function>
        </Transformation>
        <Rule>
          <RasterSymbolizer>
            <Opacity>1.0</Opacity>
            <ColorMap>
              <ColorMapEntry color="#000000" quantity="-1"/>
              <ColorMapEntry color="#0000ff" quantity="-0.75"/>
              <ColorMapEntry color="#ff00ff" quantity="-0.25"/>
              <ColorMapEntry color="#ff0000" quantity="0"/>
              <ColorMapEntry color="#ffff00" quantity="0.5"/>
              <ColorMapEntry color="#00ff00" quantity="1"/>
            </ColorMap>
          </RasterSymbolizer>
        </Rule>
      </FeatureTypeStyle>
    </UserStyle>
  </NamedLayer>
</StyledLayerDescriptor>
The performance is good enough for interactive display, and the result looks as follows:

PostGIS store improvements

The PostGIS datastore has been for years the only one that could encode a few filter functions used in filters down into native SQL, but it required a datastore creation flag to be enabled.
Starting with this release it will do so by default.


The functions supported for SQL encoding by the store are:
  • String functions: strConcat, strEndsWith, strStartsWith, strEqualsIgnoreCase, strIndexOf, strLength, strToLowerCase, strToUpperCase, strReplace, strSubstring, strSubstringStart, strTrim, strTrim2
  • Math functions: abs, abs_2, abs_3, abs_4, ceil, floor
  • Date functions: dateDifference
Along with improvements in the handling of GroupByVisitor, this makes for efficient histogram computation directly in the database, while retaining the ability to share the same code with other stores that do not have the same optimizations.

This release adds support for "array" data type in the store, with full reading and writing support, as well as native filtering (with index support, where feasible).

Finally, it's now possible to read geometries with measures from PostGIS, also thanks to JTS improved CoordinateSequence API, and encode the results in GML (the encoding is subject to a configuration flag, as GML does not officially support M encoding). The work will continue in the next few month in order to cover more formats.

Image mosaic improvements

The image mosaic module never sleeps, in this release we see the following improvements:
  • Support for remote images (e.g. S3 or Minio). In order to leverage this the mosaic index will have to be created up-front (manually, or with some home grown tools)
  • A new "virtual native resolution" read parameter allows the mosaic to compose outputs respecting a native resolution other than its native one (useful in case you want to give selective resolution access to different users)
  • Supporting multiple WKBs footprint for pixel precise overviews masking
  • A new read mask parameter allows to cut the image to a given geometry (again, helps in providing different selective views to different users)
  • Speed up NetCDF mosaics by allowing usage of stores coming from a repository, instead of re-creating them every time a NetCDF file is needed (to be setup in the auxiliary store config file, while the repository instance needs to be passed as a creation hint).
  • The mosaic now works against images without a proper CRS, setting them into the "CRS not found" wildcard CRS, aka "EPSG:404000"

App-schema improvements

The app-schema module got significant performance and functionality improvements since 19.x series, in particular:
  • Better delegation of spatial filters on nested properties to native database
  • Improved support for multiple nested attribute matches in filters
  • It is now possible to use Apache SOLR as a data source for app-schema
  • The configuration files can be hosted on a remote HTTP server

Modules moving up and down

This release sees a number of modules moving down to unsupported due to lack of maitainers, unsupported module being removed, while other modules graduate to supported status. In particular:
  • The unsupported image-collection, sfs, efeature, caching, feature-aggregate and geotiff-new module have been removed from the code base (you can still find them in older releases and version control history)
  • The OGR and GTOPO30 modules have been downgraded to unsupported status due to lack of maintainership
  • The MongoDB module moves up to supported status

Building on Windows

The GeoTools project builds again on Windows (with tests) and all new pull requests are verified using AppVeyor.

GeoTools is looking for Windows developers to join and help keep the build passing, as well as locating and squashing a few intermittent failures we're still experiencing.

Assorted improvements

Other highlights from our issue tracker release-notes:
  • Some speedups in in memory Expression evaluation, in particular attribute value extraction in simple features and equality tests
  • Support for "none" values in GeoCSS (to allow turning off symbolization/properties in override rules)
There is also a large set of bug fixes. For more information see the release notes: (20.0 | 20-RC).

Tuesday, September 18, 2018

GeoTools 20 Release Candidate Available

The GeoTools team is pleased to announce the release of GeoTools 20-RC:
This release candidate is also available from our Maven repository, and is made in conjunction with GeoServer 2.14-RC.

This release includes a various major changes.

Please Test this Release Candidate

A release candidate is your chance to both try out new features and contribute to the project with valuable feedback when we need it most.

Please test this release and let us know of any regression before we release GeoTools 20.0 final, in the second half of September.

JTS upgraded to version 1.16

JTS has been upgraded to version 1.16. This marks a significant change in package naming, the library has switched from "com.vividsolutions" to "org.locationtech"  packages and all GeoTools code has been updated to follow suit.

JTS classes are widely used in GeoTools, as a result the client code using GeoTools will also have to be modified to follow suite. Thankfully the changes amount to a search and replace and can be easily automated, please check the upgrade instructions available in the GeoTools site.

Units library migrated to JSR-363

GeoTools was using an aging and un-maintained units library and this development round moved it to a modern and maintained set of packages compatible with Java 10. Like the JTS upgrade this might require the client code to be updated, both in terms of packages and in terms of classes and methods being used.

We have provided a Units utility class, providing an easy way to match unit defined at runtime with the the correct constant.
Unit deg = Units.autoCorrect(SI.RADIAN.multiply(0.0174532925199433));
Please check the upgrade instructions available in the GeoTools site for more details.

Thanks to César Martínez Izquierdo and the participants of the OSGeo Code sprint for this work.

Various other assorted dependencies upgrades

The release upgrades many dependencies to newer versions, in particular:
  • Batik upgraded to version 1.10
  • Commons-lang to version 3.7
  • Commons-io to version 2.6
  • Mysql JDBC driver to 5.1.46
  • HSQLDB to 2.4.1
  • SQLite-JDBC to version 3.23.1
  • GeographicLib upgrades to version 1.49
These upgrades should not cause major incompatibilities, but look for for potential API changes.

Channel selection name allow expressions

GeoTools 20.x allows expressions to be used in SourceChannelName SLD elements, and in their code counterpart, thus allowing dynamic channel selection. This is welcomed news for anyone building applications that display multispectral or hyperspectral data, thus avoiding the need to build many different styles for the various interesting false color combinations.

Here is an SLD example:

<RasterSymbolizer>
  <ChannelSelection>
    <RedChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B1</ogc:Literal>
             <ogc:Literal>2</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </RedChannel>
    <GreenChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B2</ogc:Literal>
             <ogc:Literal>5</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </GreenChannel>
    <BlueChannel>
      <SourceChannelName>
          <ogc:Function name="env">
             <ogc:Literal>B3</ogc:Literal>
             <ogc:Literal>7</ogc:Literal>
          </ogc:Function>
      </SourceChannelName>
    </BlueChannel>
  </ChannelSelection>
<RasterSymbolizer>

Map algebra

This release adds support for an efficient map algebra package knows as Jiffle. Jiffle has been the work of a former GeoTools contributor, Michael Bedwards, that has been salvaged, upgraded to support Java 8, and integrated in jai-ext. From the there support has been added into the GeoTools gt-process-raster module, to be used either directly or as a rendering transformation.

The following SLD style calls onto Jiffle to perform a NDVI calculation on top of Sentinel 2 data:

<?xml version="1.0" encoding="UTF-8"?>
<StyledLayerDescriptor xmlns="http://www.opengis.net/sld" xmlns:ogc="http://www.opengis.net/ogc" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/sld
http://schemas.opengis.net/sld/1.0.0/StyledLayerDescriptor.xsd" version="1.0.0">
  <NamedLayer>
    <Name>Sentinel2 NDVI</Name>
    <UserStyle>
      <Title>NDVI</Title>
      <FeatureTypeStyle>
        <Transformation>
          <ogc:Function name="ras:Jiffle">
            <ogc:Function name="parameter">
              <ogc:Literal>coverage</ogc:Literal>
            </ogc:Function>
            <ogc:Function name="parameter">
              <ogc:Literal>script</ogc:Literal>
              <ogc:Literal>
                nir = src[7];
                vir = src[3];
                dest = (nir - vir) / (nir + vir);
              </ogc:Literal>
            </ogc:Function>
          </ogc:Function>
        </Transformation>
        <Rule>
          <RasterSymbolizer>
            <Opacity>1.0</Opacity>
            <ColorMap>
              <ColorMapEntry color="#000000" quantity="-1"/>
              <ColorMapEntry color="#0000ff" quantity="-0.75"/>
              <ColorMapEntry color="#ff00ff" quantity="-0.25"/>
              <ColorMapEntry color="#ff0000" quantity="0"/>
              <ColorMapEntry color="#ffff00" quantity="0.5"/>
              <ColorMapEntry color="#00ff00" quantity="1"/>
            </ColorMap>
          </RasterSymbolizer>
        </Rule>
      </FeatureTypeStyle>
    </UserStyle>
  </NamedLayer>
</StyledLayerDescriptor>
The performance is good enough for interactive display, and the result looks as follows:

PostGIS store improvements

The PostGIS datastore has been for years the only one that could encode a few filter functions used in filters down into native SQL, but it required a datastore creation flag to be enabled.
Starting with this release it will do so by default.


The functions supported for SQL encoding by the store are:
  • String functions: strConcat, strEndsWith, strStartsWith, strEqualsIgnoreCase, strIndexOf, strLength, strToLowerCase, strToUpperCase, strReplace, strSubstring, strSubstringStart, strTrim, strTrim2
  • Math functions: abs, abs_2, abs_3, abs_4, ceil, floor
  • Date functions: dateDifference
Along with improvements in the handling of GroupByVisitor, this makes for efficient histogram computation directly in the database, while retaining the ability to share the same code with other stores that do not have the same optimizations.

This release adds support for "array" data type in the store, with full reading and writing support, as well as native filtering (with index support, where feasible).

Finally, it's now possible to read geometries with measures from PostGIS, also thanks to JTS improved CoordinateSequence API, and encode the results in GML (the encoding is subject to a configuration flag, as GML does not officially support M encoding). The work will continue in the next few month in order to cover more formats.

Image mosaic improvements

The image mosaic module never sleeps, in this release we see the following improvements:
  • Support for remote images (e.g. S3 or Minio). In order to leverage this the mosaic index will have to be created up-front (manually, or with some home grown tools)
  • A new "virtual native resolution" read parameter allows the mosaic to compose outputs respecting a native resolution other than its native one (useful in case you want to give selective resolution access to different users)
  • Supporting multiple WKBs footprint for pixel precise overviews masking
  • A new read mask parameter allows to cut the image to a given geometry (again, helps in providing different selective views to different users)
  • Speed up NetCDF mosaics by allowing usage of stores coming from a repository, instead of re-creating them every time a NetCDF file is needed (to be setup in the auxiliary store config file, while the repository instance needs to be passed as a creation hint).
  • The mosaic now works against images without a proper CRS, setting them into the "CRS not found" wildcard CRS, aka "EPSG:404000"

App-schema improvements

The app-schema module got significant performance and functionality improvements since 19.x series, in particular:
  • Better delegation of spatial filters on nested properties to native database
  • Improved support for multiple nested attribute matches in filters
  • It is now possible to use Apache SOLR as a data source for app-schema
  • The configuration files can be hosted on a remote HTTP server

Modules moving up and down

This release sees a number of modules moving down to unsupported due to lack of maitainers, unsupported module being removed, while other modules graduate to supported status. In particular:
  • The unsupported image-collection, sfs, efeature, caching, feature-aggregate and geotiff-new module have been removed from the code base (you can still find them in older releases and version control history)
  • The OGR and GTOPO30 modules have been downgraded to unsupported status due to lack of maintainership
  • The MongoDB module moves up to supported status

Building on Windows

The GeoTools project builds again on Windows (with tests) and all new pull requests are verified using AppVeyor.

GeoTools is looking for Windows developers to join and help keep the build passing, as well as locating and squashing a few intermittent failures we're still experiencing.

Assorted improvements

Other highlights from our issue tracker release-notes:
  • Some speedups in in memory Expression evaluation, in particular attribute value extraction in simple features and equality tests
  • Support for "none" values in GeoCSS (to allow turning off symbolization/properties in override rules)
There is also a large set of bug fixes. For more information see the 20-RC release notes.