We’re excited to be at Open Source Summit in Shanghai, China from September 26-28! We have a great lineup of JavaScript speakers at the event, and we encourage you to join us on September 26. Details are below.
📅 Date: September 26, 13:30-16:30 PM
📍 Location: Shanghai Convention & Exhibition Center of International Sourcing
In July, we continued our regular work triaging and fixing Node.js security issues. We also welcomed a new contributor to the Node.js Security Working Group team, and increased the number of security releases, which improves security by making updates available more quickly. We have also continued to evaluate the Permission Model including adding a startup benchmark, adding support for V8 HeapSnapshot and Node.js reports, and cut down the number of steps it takes to create a security release. Nice progress!
As always, we want to say thank you to OpenSSF and Project Alpha Omega for their support. You can read more details about our partnership here: Security Support Role 2023.
Fixing and Triaging Security Issues
We closed 8 reports in July with 7 developers participating. Our average first response time in July was 53 hours, compared to only 3 hours in June, but we don’t expect month-to-month to always improve. It’s easy when we receive a report from contributors about the Node.js codebase, because we can quickly assess whether the report is accurate or not, almost at-a-glance. But sometimes we get reports that require a long assessment discussion before triagging the report as valid. In other words, not all reports are created equal. This elongates the process.
We also had discussions around the Node.js policy mechanism. Policies are a security feature intended to allow guarantees about what code Node.js is able to load. Some incoming issues were actually not vulnerabilities. This means people are opening issues in part because our descriptions are not clear enough. We are looking to improve in this area.
Support for Security Releases
We have started to reduce the period between security releases. 2 security releases in in the past 2 months. By shipping security versions faster and more often, it means people will get more secure versions.
For the most recent Security Release (released on Aug 9, 2023), all the processes described in the security release process were completed, including evaluating all Reports, requesting CVEs, and doing the pre- and post- Security Release announcements). The Security Release includes updates to OpenSSL.
And, two new people were interested in joining the release team. This kind of real, direct participation is great news!
We are continuing to reevaluate the Permission Model. We want to better understand how useful the Permission Model is to end users. We are getting positive feedback so far. We have also done research comparing Permission Models for other runtimes and languages. We looked at Deno, Python, Ruby, and Java. The only Permission Model similar to Node.js is Deno.
Among other improvements to the Permission Model, we added a startup benchmark – nodejs/node#48905. It shows the impact on performance of the Permission Model when it starts being used. According to our benchmark tests, the overhead is low, which is excellent.
More Permission Model improvements include:
The Permission Model Tree can be visualized by the debug environment variable NODE_DEBUG_NATIVE=PERMISSION_MODEL
Fixed Permission Model usage when using Node.js REPL
Restricted all available resources (file system, worker, child_process, inspector) when the permission model is enabled
We also are continuing to assess our security processes against Best Practices and are looking for continuous improvement on every Security WG call. This project was formerly known as the Core Infrastructure Initiative (CII) Best Practices badge. and was originally developed under the CII. It is now part of the OpenSSFBest Practices Working Group (WG). The OpenSSF is a foundation of the Linux Foundation (LF). The project was formally renamed from “CII Best Practices badge” on 2021-12-24. We have completed the Entry Level for CII-Best-Practices. For the Silver Level, there is only one question remaining! We are aiming to get to the Gold level soon. nodejs/security-wg#953
The PR to automate the security release process for security releases has been merged! nodejs/node-core-utils#665 This further automates the release process, cutting it down from 26 steps to 20. Not all steps are created equal, and the reduced steps are some big ones that took extra time. This is a huge win on the release side. And a PR has been created to automate the next Security Release issue. It is not merged but it is ready. It was used with the most recent security release. It is an “absolute significant productivity boost.”
Are you interested in getting involved? The new Permission Model is still experimental, which makes it the right time for you to try it. Be sure to join us for this month’s meetings: https://github.com/nodejs/security-wg
Talk from Luke Karrys, Senior Software Engineer at GitHub at OpenJS World 2023 in Vancouver, Canada, May 10-12.
The npm CLI team manages almost 100 different projects that account for 4,000,000,000 downloads per month. And the best part is all of it is open source! Each project includes automated releases, open bug bounties, triage for community issues and pull requests, (almost) full test coverage, and is all managed by a team of four engineers.
In this talk, npm CLI engineer Luke Karrys covers the tooling and processes that allow the team to confidently and securely ship new releases every week for the CLI and some of the most used packages in the JavaScript ecosystem including Semver and which. In the talk, Luke details lessons the team has put into practice from their collective decades of open source experience.
Talk from Stephen Husak, Distinguished Engineer, Capital One at OpenJS World 2023 in Vancouver, Canada, May 10-12.
Stephen Husak shares insights on how a large enterprise manages the risks associated with the constantly evolving vulnerability landscape. The talk begins with an overview of the security landscape in the JavaScript ecosystem. It then delves into how Capital One mitigates risks by adopting well-managed and purposeful practices when utilizing open source software.
Stephen goes into more detail on how this is done in partnership with Capital One’s Open Source Program Office and subject matter experts across the company. Stephen describes how Capital One utilizes a working-group model as well as using process, governance, and automation tools to minimize risk and reduce developer toil. He promotes responsible usage of Node.js and its associated modules. The talk concludes with a Q&A session and Stephen provides additional resources.
Talk from Kazuhito Yokoi, Software Engineer, Hitachi, Ltd. at OpenJS World 2023 in Vancouver, Canada, May 10-12.
To promote the use of cloud services or devices from Node-RED, companies can easily publish their original connectors to the Node-RED flow library. But creating custom connectors is a time-consuming task because it requires coding with Node-RED-specific development rules. To solve this situation, Hitachi developed the Node generator tool as one of the Node-RED projects under the OpenJS Foundation. This tool can convert to custom connectors from various sources like OpenAPI documents. Recently, this tool has supported generating custom connectors from subflow as a new source. Using the subflow functionality, all Node-RED users are able to create their original connectors from the existing Node-RED flow without coding. In this talk, Kazuhito shows how to use the tool and integrate it with GitHub Actions to release connectors to the public semi-automatically.
The Sovereign Tech Fund, financed by the German Federal Ministry for Economic Affairs and Climate Action, is providing the OpenJS Foundation with EUR €875,000 (USD $902,000) in government funding to strengthen JavaScript infrastructure and security.
We’re off to a quick start! 🏃♀️💨
In 2023 Q2, our main challenge was to quickly establish a cross-functional project within the Linux Foundation with enough resources and processes in place to complete the Q2 deliverables. It was also imperative to communicate with our projects to create momentum for the work.
In a condensed Q2, we ramped up quickly and briefed our worldwide stakeholders including our JavaScript projects top maintainers and contributors in their security, build and release teams. Approximately one-third of our projects immediately signed on to participate in the program. And we’ve completed initial surveys on infrastructure and security with these projects.
As a result of this early momentum, we are well-positioned to accomplish our Q3 goals.
Program Management Key Accomplishments
Core JavaScript project team and stakeholders identified and onboarded
All Project Management program components created including: task tracking, recurring meetings, project inventory, reporting templates and communication channels
Financial framework for tracking and reporting implemented with Linux Foundation CFO
Project briefing deck created, and four multi-project onboarding meetings were held to accommodate schedules and timezones. 1:1 briefings were held for others
Infrastructure Key Accomplishments
Project inventory form developed and sent to projects
Analysis of inventory responses completed
Proposed solutions ready for internal review
“Project completion” defined
Security Key Accomplishments
Hired and onboarded security engineer
Selected audit and training vendor
Inventory and initial analysis completed for project audit priorities
Prioritized list of projects socialized with Security Collab Space
Scope of Badge Program and Secure Releases/CVE management defined
We believe we are off to a great start. There’s much more work to be done. If you are involved in open source software development and are interested in finding out more information about our efforts, please feel free to contact us at info@openjsf.org.
Talk from Darcy Clarke, Open Source Engineer, Independent at OpenJS World 2023 in Vancouver, Canada, May 10-12.
Darcy Clarke, an independent open source engineer, highlights the constant threats and attacks faced by the software supply chain, with a particular focus on the JavaScript ecosystem. The talk explores the current state of the ecosystem, emphasizing the importance of managing dependencies, including transitive dependencies, and the various threats to the software supply chain. Darcy also shares insights using the “Create React App” project as an example.
The presentation emphasizes the key factor of accuracy in securing the supply chain and provides practical advice, including avoiding mutable package references, using lockfiles, and caching and bundling dependencies. Darcy then discusses the existing solutions and tools available, such as security companies, advisory tools, software bill of materials (SBOMs), cryptography, scorecards, and badging. Future state solutions and tooling are also explored, focusing on introspection and validation. The session concludes with a short Q&A session and key takeaways.
In June, we saw all of our Node.js security metrics trending in the right direction. Closed reports were up, average first response time was down (again), and much more. Our Threat Model is now being used regularly to help assess issues. And we are getting comments on our Security Model, which is the kind of interaction that makes processes robust. We’re not claiming victory, but this feels like progress.
As always, we want to say thank you to OpenSSF and Project Alpha Omega for their support. You can read more details about our partnership here: Security Support Role 2023.
Fixing and Triaging Security Issues
The Node.js team closed 17 reports in June which is a big increase from the 2 completed in May. We don’t expect the number of reports to increase linearly, but this still qualifies as a good month for improving Node.js security issues.
Also, Node.js team’s average first response time in June was 3 hours, compared to 8 in May. Remember our goal is average first response within 48 hours, so this is excellent. We’d like to extend special thanks to Tobias, Bradley and Rafael for their help as volunteer triagers!
A lot of effort was made to include all the fixes on time for the Node.js security release that went out on June 20, 2023. Last year, security releases came out about once per quarter, which was not frequent enough. We are looking to increase the frequency this year.
Support for Security Releases
Security Release coordination continues to improve. All the processes described by the security release process – multiple steps for planning, announcement one week in advance, and release day – were completed.
One big improvement is automation. For each security release, there used to be 26 steps and then 12 steps for the release itself. But with the OpenSSF investment, we have been able to dedicate time to automate, establish new processes, and streamline the workflow. Each version required all those steps (v20.3.1, v18.16.1, and v16.20.1).
The most recent Security Release included updates of two Node.js dependencies: OpenSSL and c-ares. All the releases were sponsored by OpenSSF.
And there was one regular release of Node.js v20.3.0!
Node.js Security Working Group Initiatives
The Security Working Group is making progress on the 4 main initiatives for the Security Working Group Initiatives for 2023: Permission Model, Automate update dependencies, Assessment against best practices, and Automate Security release process.
For the Permission Model, 5 security fixes for CVEs were completed. Regular fixes and pull requests were also addressed.
The Security WG is actively looking for more feedback. If you are interested in helping to define the initiatives, please participate!
Automated Update Dependencies
The initiative has been completed, it was just missing backports. It is now ready to be merged! 🎉
Assessment Against Best Practices
The Security WG is continuously looking at best practices and doing improvement on each Security WG call. One area of effort is CII-Best-Practices for Node.js Projects. Node.js looked at this early, 7 years ago, which means we were forward looking, but it needs to be updated.
Automate Security Release process
A PR has been created to automate the release proposal for security releases. The Security Release proposals were created using this automation
Good discussion around assessing experimental features. Should experimental features be assessed in the same way as stable features? Currently under consideration.
Improving Security Processes
There is a new PR now to help create security issues. It automates GitHub issue creation. It should eventually manage all states of a security release. The PR includes a new command CREATE and there will be other PRs to manage steps beyond CREATE, such as requesting CVEs, creating issues, sending emails and more.
Are you interested in getting involved? The new Permission Model is still experimental, which makes it the right time for you to try it. Be sure to join us for this month’s meetings: https://github.com/nodejs/security-wg.
Talk from Abigail Cabunoc Mayes, Program Manager, GitHub at OpenJS World 2023 in Vancouver, Canada, May 10-12.
Abigail Cabunoc Mayes delves into key aspects of supporting and maintaining open source projects. The talk covers various strategies to ensure the sustainability of projects, such as providing financial support to project maintainers and implementing succession planning practices. Abigail highlights the advantages that corporate open source initiatives have in terms of hiring dedicated maintainers.
The importance of succession planning for open source projects is also emphasized, given the steady increase in both open source adoption and contributors. Abigail then presents a case study involving past collaboration with Mozilla Open Leaders, discussing the implementation of payment mechanisms for maintainers and metrics to track financial practices. Additionally, the talk offers practical tips and guidance for others to adopt and practice sustainable open source software, focusing on community engagement, financial support, and engineering practices.
“OpenVis” is a collaboration space and a forum within the OpenJS Foundation to neutrally govern kepler.gl, deck.gl, and the vis.gl suite of frameworks, a comprehensive and widely adopted set of visualization libraries based on JavaScript and WebGL.
Some of the milestones in the past year include:
👥 kepler.gl is now used by around 30,000 weekly users (for the application version) with a wide range of industry integrations (for the library)
📈 deck.gl is one of the top web-based visualization libraries and it doubled its growth to over 136,000 weekly downloads
Under the OpenJS Foundation, OpenVis has flourished with open governance. Open governance not only embodies the principles of open source, but it further enhances them. While the source code is available like with traditional open source projects, open governance takes it a step further by ensuring decisions are made collectively. Contributors openly discuss, collaborate, and cooperate, driving the direction of the project in a transparent manner. In the past month, a new website was created to support the project!
Project Growth
kepler.gl and the vis.gl frameworks are all part of OpenVis. vis.gl is a suite of frameworks for GPU powered data visualization and analysis of large datasets on the web. It is one of the most widely adopted WebGL visualization libraries, with close to 100K daily downloads from npm. The offerings of vis.gl are packaged and best represented by its flagship framework, deck.gl. And kepler.gl is a data-agnostic, high-performance web-based application for visual exploration of large-scale geolocation data sets.
In the past year, OpenVis accomplished a wide range of improvements and upgrades:
Added TypeScript to kepler.gl, deck.gl, loaders.gl, and luma.gl.
And a lot more
By any measure, it was a fantastic first year. And we have big plans coming up!
Hitting Big Milestones
kepler.gl
kepler.gl stands out as one of the most powerful open source browser-based tools for geospatial analysis and visualization of large data sets. Built on top of the deck.gl and vis.gl frameworks, this web-based application is designed for exploratory geospatial visualization. The beauty of kepler.gl lies in its powerful and intuitive UI, allowing both technical and non-technical users to visualize data with ease. Moreover, it’s free and requires no sign-up, making it available as both an application and a UI library.
Recognition and Use Cases
kepler.gl is well known in geospatial analytics, visualization fields and the mobility space, with approximately 30,000 weekly users. It’s been integrated with a variety of platforms, including Jupyter Notebooks, Jupyter Labs, VSCode, Tableau, and Apache Superset. Many companies in the mobility space use kepler.gl internally for geospatial analysis, demonstrating its versatility and efficiency.
Integration and Customization
Companies, including Foursquare, Uber, and CARTO, have adopted the kepler.gl UI library for creating their own customized applications. Recent updates have focused on “hardening” kepler.gl, making it more robust for production applications. These improvements encompass conversion to TypeScript, modularization, exposing more APIs, and numerous fixes. React component factories, which allow the injection of custom components into the UI and handle state changes, have been improved. A key advantage of using the kepler.gl UI library is the reduced need to fork kepler.gl, avoiding long-term maintenance challenges.
Conversion to TypeScript Reduces Complexity
By far the biggest change in kepler.gl is the conversion of the entire code base to TypeScript, involving over a person-year of work. This conversion was aimed at mitigating code base complexity. This means that:
Developers can look up type definitions to quickly understand what the expected data formats are in various cases.
Developers now have a strong safety net when making changes and additions to the code
Smaller Modules Helps Reduce Size
The kepler.gl code base keeps growing. To help developers, big monolithic modules were broken up into independent smaller ones published on npm. Developers can install only what they need.
React-Map-GL: Support for Alternate Basemap Libraries
The base map library in kepler.gl is available as its own React component. React-Map-GL is a user-friendly API wrapper for React. It works with Mapbox and now MapLibre. Version 7, released this year, was a complete rewrite of the library, addressing issues in versions 6 and 5. The bundle size has been reduced by 74%. Support for any Mapbox-compatible plugins like mapbox-gl-draw and mapbox-gl-geocoder, to name a few, has been added and has paved the way for adding compatibility for more map libraries.
Later this year, OpenVis plans to add a Google Maps React wrapper which will function similarly to the existing wrappers.
vis.gl and deck.gl
vis.gl is a suite of JavaScript visualization frameworks. The offerings of vis.gl are packaged and best represented by its flagship framework, deck.gl.
deck.gl is one of the top web-based visualization libraries, with over 136,000 weekly downloads, doubling its growth compared to last year.
Integration and Application
deck.gl has been integrated with most popular base map providers such as Mapbox, Google Maps and ArcGIS. Its compatibility extends to bindings for React, Python/Jupyter, R, Vega and CUDA, making it a versatile tool for various applications. deck.gl also offers libraries for specific applications such as 3D geometry editing (nebula.gl), animation (hubble.gl), autonomous vehicles (AVS), multiplexed bioimaging (Viv), to name a few.
Companies including Google, ESRI, CARTO, Foursquare and Cesium have contributed to deck.gl, enhancing its capabilities to work with their libraries or data formats.
Complex Visualizations Made Simple
deck.gl is an ideal tool for exploring and visualizing large datasets. For simple projects with maps, a user might just use react-map-gl. But for more complex and customized visualizations,, deck.gl’s extensive catalog of composable layers, combined with facilities for creating custom layers take applications to the next level..
It also makes it easy to package and share those visualizations as reusable layers for other people. While the deck.gl API follows a Reactive programming paradigm, making it work seamlessly with frameworks such as React, deck.gl is a pure JavaScript framework, and works in any environment that supports WebGL.
Enhancing Development and Publishing Tools
The tools for deck.gl development and publishing have also seen significant improvements. All examples are now bootstrapped with vite, pre-building scripts have been updated to use esbuild, and the website documentation is generated with Docusaurus. All of this allows first-time users to get started more quickly, for either contributing or just using the library.
Improving Robustness with TypeScript
Like kepler.gl, deck.gl was converted to Typescript. This conversion was not aimed at making the framework more developer-friendly, but also at improving the robustness and maintainability of the code, making outside contributions more manageable.
New Layer Extensions
A focus of developers for deck.gl over the past year was Extensions. Extensions can be optionally added on to core deck.gl layers without bloating the core. They are not included in layers by default.
There are currently 9 Extensions available and developers can author their own layer extensions. Three new extensions were added in the last year – MaskExtension, CollisionFilterExtension, and TerrainExtension:
MaskExtension – Allows layers to show/hide objects by a geofence. The masking is performed on the GPU
CollisionFilterExtension – Allows layers to hide features which overlap with other features. Works with all layers within the library, like text, scatter plot, and more.
TerrainExtension – Renders otherwise 2D data along a 3D surface. geoJSON can be overlayed on an elevation model. This is especially useful when viewing a mixture of 2D and 3D data sources. The repositioning of all the geometries is done on the GPU, so it is done dynamically in real-time and interactively. The designer of the maps does not need to focus on the complexities of offsetting the 2D and 3D maps.
Photorealistic 3D Tiles from Google Opens Up Opportunities
Photorealistic 3D Tiles was released by Google with a dataset that is comparable to Google Earth. Users can now leverage the deck.gl Tile3DLayer to render entire cities in amazing detail. Combining this with TerrainExtension allows overlaying 2D layers onto 3D cityscapes. All of this can be done at runtime with very little code. See documentation for getting started.
This opens up a huge opportunity for exploratory analysis capabilities. It’s more than just a technology advancement. Instead of unique solutions from different vendors like Google and Mapbox with their own distinct visualizations, deck.gl’s open governance model and OpenVis standards can connect these diverse solutions, and lead to a more collaborative and integrated mapping ecosystem.
Special thanks to OpenVis members at CARTO and our Technical Steering Committee for help in these areas.
Project Highlight: Add Lighting and Effects for Stunning Results
Community member, Chee Aun Lim, demonstrated the remarkable creative potential of deck.gl in a captivating demo. By skillfully employing the built-in Effects system, Lim incorporated Sun Lighting and Shadows to lend depth to his data visualization. This was further enhanced with the application of Post Processing Effects, resulting in a polished, visually impressive representation. This project is a great example of how data visualizations can be transformed into immersive experiences. We highly recommend exploring this project on Github!
New Framework flowmap.gl Joined OpenVis
flowmap.gl is a framework for geospatial flowmaps. It is a JavaScript module which can be used for visualization of geographic movement: mobility, transportation, migration, and more. For flows like people moving around a city, or a subway system, you want to know the location but also see how data changes over time. The layer is rendered in a WebGL context and is capable of adaptive aggregation and filtering, which allows it to handle relatively large numbers of flows. Flowmap.gl is adding a variety of deck.gl layers for flow data.
Community Growth and the Open Visualization Collaborator Summit, Madrid, Spain, Oct 2022
OpenVis recently held the first ever Open Visualization Collaborator Summit with about 100 contributors participating from a broad international open visualization community. Participating companies included CARTO, Google, Joby Aviation, Microsoft, Foursquare, Mapbox and many more.
There was a great lineup of talks and presentations. Just two key examples:
Paul Taylor, NVIDIA, on “GPU-accelerated Geospatial Analytics with NVIDIA RAPIDS,” showed how it is a lot easier to optimize data analysis and visualizations with the latest CUDA GPUs. Before, deck.gl performance was limited by web browsers, since it’s a JavaScript library. Now users can use deck.gl with the latest native desktop APIs within Node.js. This lets users render much more data much more quickly.
Kyle Barron, Foursquare, on “GeoArrow and GeoParquet in deck.gl” showed how to use GeoArrow and GeoParquet in JavaScript in the geospatial stack. He did it with deck.gl, which offers a low-level binary interface for data-intensive applications. Writing a custom binary implementation for day-to-day applications can take too much time and effort. With GeoArrow and GeoParquet, it can be done with a couple lines of code. Users can use Node.js and desktop rendering environments. This is continuing to push what can be done in the browser without any special graphics hardware.
A second Collaborator Summit will be held in September 2023 in New York City. You can register here to attend. Speakers will be announced in the coming months.
The Future of OpenVis
Corporate members have helped fund OpenVis progress. In particular, last year when the OpenJS Foundation partnered with the Urban Computing Foundation (UCF) to form the Open Visualization Collaboration Space, we welcomed four UCF members into the OpenJS Foundation: Foursquare, HERE Technologies, Joby Aviation, and Uber. Open visualization technologies are core to each of these companies’ leadership positions in the market, and they are energetically supporting the infrastructure and long-term growth of OpenVis. We wanted to extend a special thank you for their support and commitment this past year, and look forward to continued progress.
On the technology side, WebGPU has just come out in Google Chrome. It is the next-generation web API for accessing GPU resources. It’s a big departure from OpenGL that WebGL is built on. Right now, deck.gl and luma.gl are all on V8. V9 is scheduled to add WebGPU support to luma.gl and, therefore over time, deck.gl.
WebGPU changes the shader language that is being used and a lot of libraries will need to be updated all at the same time, so this process will take time.
With luma.gl, you will have a standardized interface for accessing either WebGL or WebGPU for rendering or GPU access. This is a good process. In the beginning deck.gl will continue to use the WebGL path that it already has, but over time that will change. We want a smooth transition from WebGL to WebGPU.
There is enthusiasm for WebGPU. Ultimately, OpenVis wants deck.gl to be a flagship WebGPU library. But it will require a lot of libraries to support WebGPU at the same time. Interleaved rendering between libraries takes lots of time and development effort.
Get Involved
We appreciate all of our contributors who have participated in the OpenVis Collab Space this year. We look forward to many years ahead!
Interested in getting involved? Join our bi-weekly community meetings to collaborate and learn all about OpenVis. Details on the OpenJS Foundation Public Calendar.
Shoutout to Chris Gervang for detailing these great milestones for OpenVis. You can watch his talk from OpenJS World 2023 on YouTube now.
