Version 3.0 of kepler.gl is now available, bringing new geospatial features to kepler.gl application users and dramatic improvements to developers using the kepler.gl libraries.
By Ib Green, Foursquare and Shan He, Foursquare
The post-processing effects feature from Foursquare Studio has been upstreamed into the open source kepler.gl, bringing lights and shadows, and a library of image processing effects.
Lights and shadows, depth-of-field and other effects in play.
For more information, see Elevate Your Maps with Foursquare’s Artistic Geospatial Tools, which describes the feature set in Foursquare Studio that has now been merged back to kepler.gl.
kepler.gl 3.0 starts adding support for the Apache Arrow format, enabling dramatic increases in loading performance and maximum dataset sizes.
In this release, kepler.gl can load GeoArrow files, which can be visualized with the PolygonLayer. See the roadmap section below for more general Apache Arrow file support and support for other layers.
10x faster! Loading 1 million polygons takes only ~2 seconds with GeoArrow format
vs. ~20 seconds with GeoJson format.
Incremental loading of 11 million row GeoArrow dataset
The Apache Arrow tables support all features, including filtering and tabular viewing.
The trick behind the massive performance boost is that Apache Arrow files are loaded as binary data. Binary data loads
Binary data support in kepler.gl has been a long time in the works and required deep refactoring of the kepler.gl core logic. For some additional information, see the DataContainer bullet in the kepler.gl library updates below and the roadmap.
Substantial parts of the GeoArrow support was contributed by the Center for Spatial Data Science at the University of Chicago.
A new Foursquare cloud plugin is available in kepler.gl, letting users save, load and publish kepler.gl maps via their Foursquare accounts. It also enables users to open kepler maps in Foursquare Studio, from which additional functions such as sharing, web publishing, and much more is available.
Foursquare is building a full suite of freemium tools on top of kepler.gl, and kepler.gl can now be used as a part of the Foursquare Studio platform through this new integration.
Foursquare accounts are free to create, offer a gigabyte of free cloud storage, and give access to a wide range of additional geospatial tools.
The Foursquare cloud plugin is an addition to the existing CARTO and Dropbox plugins, giving the user a wider range of options for how to store kepler.gl maps.
Naturally, cloud plugins are optional to use, and none of your data is shared with any cloud provider without the user’s explicit login and save action.
The default base map renderer has been changed from Mapbox to Maplibre to ensure that kepler.gl is using an open source basemap that is actively supported by a strong community.
The maplibre integration was provided by Birk Skyum.
Apart from the headline features listed above, users may want to be on the lookout for numerous small performance and usability improvements in areas such as:
The vast majority of kepler.gl 3.0 development work has taken place under the hood, improving the kepler.gl frameworks that power all kepler.gl based applications.
kepler.gl is a framework that lets developers quickly build their own customized tools for visual geospatial analytics. In fact, the popular geospatial analytics application hosted at https://kepler.gl is simply a small demo application built on top of the kepler.gl framework.
The entire kepler.gl code base has been converted to TypeScript. This helps mitigate code base complexity and has increased developer velocity:
The kepler.gl libraries are published as a suite of namespaced npm modules. This makes kepler.gl more modular and composable, and enforces stricter separation of functionality, which should help both developers and maintainers.
The DataInterface lets developers change how data tables are stored and organized internally, with having to modify all the places where.kepler.gl accesses table data.
The data container was developed by Foursquare.
Many additional React UI components now export “factories” making them overridable.
When building applications on top of the kepler.gl library, it is often desirable to override components and behaviors of kepler with custom implementations.
Kepler.gl 3.0 also upgrades a number of its dependencies to the latest versions. This brings in a wide range of bug fixes and performance improvements. Examples of upgraded dependencies are:
kepler.gl is under open governance in the OpenJS foundation, and for the last few years kepler.gl has been maintained by Foursquare, where the original creators are using kepler as an open core for building location platform products such as Foursquare Studio.
While it took a long time for kepler.gl 3.0 to be released, development has never stopped. More than 800 commits have been made to the kepler.gl master branch after the release of kepler.gl v2.0 in February 2020. Though admittedly most of these commits have been focused on strengthening the foundations of kepler.gl rather than new features.
The current goal of the kepler.gl team is to increase release cadence, ideally to a quarterly release and also increase the amount of end-user facing geospatial features in each release.
If you’d like to participate in kepler.gl discussions or if you would consider contributing in any way, it is easy to engage with the kepler team through the Open JS Foundation at https://www.openvisualization.org/.
As mentioned above, the kepler.gl developer team would like to ship quarterly releases. Some of the major items we hope to work on for those release are:
Adding Apache Arrow support to all layers (beyond the PolygonLayer), and extending binary data support to other file formats, so that users can benefit from massive performance gains in more situations.
An exciting trend in the geospatial industry is the momentum around cloud-native geospatial file formats. We’d like to keep adding support for loading from and exporting to these formats, including GeoParquet, FlatGeobuf, PMTiles, etc.
Kepler.gl builds on deck.gl, and deck.gl v9 development is focused on WebGPU support. Upgrading kepler.gl to deck.gl v9 once it becomes available will bring opportunities to build even more performant and capable GPU visualizations and compute capabilities.