Community office hours
Background
Every week we host a live office hours session where we talk about the latest changes that are happening, answer community questions, and teach you more about specific topics.
You can find each of the office hours sessions below along with a summary of what was discussed and the key links you might find helpful.
OSS contributor conversation with Shannon Pamperl (April 24th, 2024)
Summary and related links
Announcements for the week:
Releases:
We've done a new full release of
rewrite
and therewrite-recipe-bom
. For a full list of changes, check out our changelog over on the OpenRewrite docs. Special shout out to the folks over at IBM who contributed a ton of changes on therewrite-migrate-java
project.
Events:
Tim will be at JCON Europe from May 13th - 16th. He will be giving a talk on Transforming Code with OpenRewrite and Refaster and Weeding your Micro Service Landscape. He has a limited amount of free tickets to give away if you'd like to attend and are going to be in Germany at that time. Please reach out to him to learn more.
Tim and Jonathan will be at Spring I/O from May 30th - 31st in Barcelona. They'll be giving a talk on automated software refactoring with OpenRewrite and generative AI.
Jonathan and Olga will be at UberConf in Denver on July 16th - 19th. They will be giving two workshops plus a keynote presentation!
Tim gave a short summary of what happened at Devnexus (one of the conferences he recently attended). We were especially proud of the fact that many other speakers mentioned us and OpenRewrite in their presentations. Our AI talk was also packed with people wanting to learn more – which was exciting. We also learned some fascinating things from the community such as the fact that some places are still looking to update from Spring 1 and 2. If you're one of those places, make sure to check out our Spring Boot 3.x recipes that can help!
Content:
Justine recently put out an excellent blog post on safe, efficient AI for mass-scale automating refactoring. If you're interested in learning more about how Moderne uses AI, definitely check it out.
We've been collaborating with MILA on AI research. This past week was especially exciting in that regard as Yoshua Bengio, the founder of MILA, was named as one of the 100 most influential people in 2024. Congrats to him!
We released a new video this week on how to find sensitive API endpoints in your services.
If you missed it from last week, Jonathan was recently on an airhacks.fm podcast talking about the history of OpenRewrite.
Press/Mentions:
OpenRewrite has joined the Commonhaus Foundation. The Commonhaus Foundation is a new model for established open source libraries and frameworks seeing a neutral home. The non-profit's mission is to ensure a place where open source projects can continue to thrive for future generations. This shows our commitment to OpenRewrite remaining safe and open source even if something were to happen to Moderne. For more information, check out the press release.
Netflix featured us in their presentation at Devnexus where they were talking about modernizing and cleaning up their code.
We then introduced Shannon Pamperl. He's one of the oldest and most prolific contributors to OpenRewrite. We then asked had a lengthy conversation with him on his history with OpenRewrite. Some of the questions that Shannon answered include:
How did you get started with OpenRewrite?
Knew Jonathan from a previous job and always respected his work. Got a message from Jonathan and began looking into the project right away.
What were some of the problems you used OpenRewrite to solve?
Wanted to use it with Gradle. Helped kick off discussions that led to
rewrite-groovy
andrewrite-gradle
.
What are some of the challenges you had when learning to use and create recipes for OpenRewrite?
Biggest challenge was getting a grasp on the data model itself such as "What is an LST?" or "What is a Compilation Unit?" (If you want to learn more about these, check out our Java LST examples doc where we provide visualizations and break each LST type down). Another complexity was understanding the visitor pattern and how that worked.
If you want to learn more about LST types, check out our Java LST examples doc where we provide visualizations and break each LST type down
If you want to learn more about visitors, check out our documentation on visitors that also includes some videos we've made on the topic
Were there any particularly challenging contributions you made?
You've been working on Gradle Kotlin support - how has that been going?
For both of these, difficulty has been with type attribution.
Happy to have help with this project. If you want to, consider contributing to rewrite-gradle-experimental.
We've found that Shelter engineers are extremely competent and have technical depth in a variety of fields. Is there something that Shelter does right that helps nurture great engineering talent?
Shelter stands by their engineers. They push you to do your best while also understanding that people don't know everything. When they don't know something, they'll work together to figure it out without judgement. Every individual person is taught and levels up over time – which compounds when there's a team of people like that.
If you could make a wish right now on one thing OpenRewrite or Moderne would do that it doesn't right now, what would that be?
More parsers for more languages. TSX would be super valuable. TypeScript/React/HTML compounded together can be tricky to handle.
Are there any tips/tricks you have that newer recipe authors would benefit from?
Focus on doing one thing really well. When making a recipe, make it simple and focused and ensure that it completely solves the core issue. Then, you can combine multiple of these recipes together to make something more substantial – and you can be confident in the individual pieces.
Are there any libraries or common platforms that you wish there was a recipe for?
Most libraries that he uses already have recipes for them, but they lack features in them. When he finds issues that are important, he tries to contribute back recipes for them.
If someone wanted to get started with OpenRewrite, what would you recommend?
Start by coming at OpenRewrite with a problem that you want to solve, so you have some context. Then, join the OpenRewrite community Slack and ask questions in there. He'd also recommend looking at RewriteTest and rewriteRun to get a better idea of the flow of recipes.
(We'd also recommend checking out our documentation where we've created numerous tutorials and references to help people get started. If you run into any issues or wish for something to exist that doesn't, please let us know!)
Could you tell us about the challenges with using feature flags that motivated you to make contributions to LaunchDarkly recipes?
At big enterprise companies, it can be very hard to keep up with the open source community as it moves so fast. Feature flags may get added and then forgotten about. Having recipes to find and remove these has been super helpful.
Are you using LaunchDarkly recipes to get rid of feature flags at Shelter?
Yes. It's much nicer to use recipes than manually doing it across systems.
Is there anything cool that we should have asked that we missed?
Shannon was curious what's going on with cycles in relation to recipes. Sam dove into some of the issues that we've run into with trying to eliminate cycles.
Is there anything missing from the recipe execution lifecycle?
Unsure. Biggest problem right now is ScanningRecipes behaviour when there are multiple of them trying to interact with a single top-level LST element. Mentioned that the individual elements just works but that the combination causes problems.
Is there a way to make it easier to compose ScanningRecipes together?
Probably. Not sure on exact details. If you (the person reading this) have any ideas, definitely let us know!
Recipe authoring tips (April 17th, 2024)
Summary and related links
Announcements for the week:
Jonathan Schneider (our CEO/co-founder) was recently on the Front Lines podcast. They talked through topics ranging from his transition from the military to tech to how Moderne approached getting our first paying customer to what his vision is for the company and the future.
If you want to learn more about the history of OpenRewrite, check out this airhacks.fm podcast with Jonathan Schneider.
We released a blog post detailing how to deploy LLMs where GPUs are expensive and/or just not available. If you're looking to operationalize an LLM in a cost-effective way, definitely give it a read!
We also released a blog post detailing how OpenRewrite and Moderne can streamline feature flag management so that you can reduce the risks associated with using feature flags in production code.
On the video side of things, we released a video about creating AI visualizations with Moderne.
We then jumped into the main topic for the week – tips and tricks for recipe development:
We began by looking at the Gradle Test use JUnit Jupiter recipe, which, on paper, seems like it should a relatively straightforward recipe. The core idea behind it is we want to
useJUnitPlatform()
to theTest
task configuration for Gradle projects. It's only a line or two of changes - but there are many cases to consider. For instance, the existing project may not have any test tasks set up, in which case you would need to add that along with the changes. Other projects may already have a test task set up, and they are stylized in a certain way, in which case you would want to match their existing style and add the configuration alongside of it.We then went through a pattern we use when developing recipes like this. We start off by ensuring we don't make changes when they're inapplicable or when we can't be confident enough they would be correct. We then try and find cases where no change is required as the change has already been made. This helps ensure the recipe isn't making any unnecessary changing and also helps ensure we aren't wasting processing time or power when it's not needed. After that, there are several layers of attempting to update existing configuration. Only after everything else has been handled, do we attempt to add new configuration.
The core idea with this pattern is to start off with the smallest pieces and work your way up to the most complex. This not only helps improve the speed of the recipe results, but it also makes it easier to create tests and gain confidence as you go.
Another important note with this recipe is that we created multiple visitors that are used throughout the base part of the recipe. This helps improve readability as the core flow through the recipe is greatly simplified. This also helps improve testability as you can test more focused visitors or paths.
We then dove into the individual visitors to help explain what's happening and why. One notable caveat with Gradle visitors is that we can't be as confident in types – so we have to perform some additional checks to ensure that we are parsing/modifying the correct thing.
The next topic was talking about the difference between a traditional, imperative recipe and a scanning recipe. The core idea being that regular recipes can not be used if you need to take information from one file to modify something in another file as they only look at each file once in a non-configurable order. Scanning recipes, on the other hand, get access to different phases of the recipe lifecycle (such as the scanning phase which comes before the edit phase and allows you to look through all the files, but not make any changes to them).
We then dove into recipes that are composed of multiple scanning recipes – such as the AddDependency recipe. We talked through some things you'll need to think about and be aware of when writing your own scanning recipes – such as ensuring scanners are created for each recipe you want to use.
AI integrations at Moderne (April 4th, 2024)
Summary and related links
This week we welcome Justine - our AI/machine learning research engineer!
Announcements for the week:
DevNexus is next week! If you're going to be there come say hi.
We released a new video detailing the Moderne DevCenter.
To go along with that, we also have a getting started guide for those of you who are Moderne customers and want to jump in.
AI topic:
We started off by talking through search in Moderne. Prior to AI search being added, it was very hard to discover recipes, typos would result in no recipes being found, and words that were tangentially related did not provide results. Adding AI search helped fix all of these issues.
We then provided some more context into how AI search works and how you might get started with your own. Check out Hugging Face to look through various models - which includes the ones we used.
We talked through common questions about these models such as how they work, how they're trained, what type of system you need, etc.
If you want to learn more details about AI at Moderne, Justine also wrote a lovely blog post on fast, secure, and cost-effective AI searching.
We then went over some diagrams that Justine made to explain how AI works across Moderne.
Next up was diving into various types of visualizations and how they can be used to discover important elements in your code. For instance, we highlighted a visualization that shows the different languages that are in use in your comments - and one that clusters method names together.
We provided an example of how we used AI to help one of our customers who had mis-encoded French comments throughout their code base. This made it a struggle to automate some key things they wanted to accomplish such as documentation generation.
We concluded by talking through some upcoming AI additions to Moderne. One of these is a way of using AI to search through code. For instance, if you had a general idea of what you wanted to search for but weren't quite sure the exact thing to search for, you could use this to find related code.
Another upcoming change is using AI to make suggestions of recipes to run on your code base to make it more secure. For instance, if it saw you were using an improper random number generator, it might recommend running a recipe to change it to use a secure one. These recipe suggestions will appear in the DevCenter for the organization.
Dependency management recipes (March 25th, 2024)
Summary and related links
Only a few announcements this week since it's so close to the last office hours:
We will be presenting on Security Boulevard at 11 AM ET on March 28th, 2024. In that presentation, we'll discuss how to massively pay down your tech debt fast.
As a reminder, we will be at DevNexus 2024 in April.
The main topic was about dependency management:
We talked about how to gain insight into your dependencies. For instance, maybe you're curious about if all of your repositories are using the same version of Jackson across all of their modules.
You can use the visualizations or data tables produced by this recipe to get detailed information about that.
Once you've dug into what dependencies you're using across your repositories, you may want to upgrade your dependencies to be on the same version.
This is also a really useful recipe to use when writing migration recipes such as newer versions of Spring.
The code for both of the above recipes can be found in the rewrite-java-dependencies repository.
Another common thing developers need to do is update their Gradle wrapper as new versions of Gradle are released. We demonstrated how you can use the Update Gradle wrapper recipe to update this across all of your repositories very quickly.
We highlighted some recipes that were enhanced recently to help with dependencies. For instance, in the change Maven parent recipe, we've added support so that properties that used to exist in the parent pom (but don't anymore) can be brought down into the child pom so that the project can keep compiling until you can upgrade it.
To go along with the above recipe change, we also updated the Remove redundant explicit dependencies recipe so that you can remove dependencies that exist in both your current pom and the parent pom – with options to let you decide if you only want to do that when the versions match exactly or not.
As part of investigating dependencies, it's a good idea to check for and possibly fix vulnerable dependencies.
We concluded by highlighting the dependency tree generated by the dependency report recipe. This is a searchable tree that helps you answer the question of, "What do I have that includes this dependency?" or "What uses this specific version of this dependency?"
Running JavaScript codemods at scale (March 20th, 2024)
Summary and related links
As usual, we began by discussing announcements related to OpenRewrite & Moderne:
Jonathan and Olga will both be presenting at DevNexus 2024 in April.
Jonathan will also be giving a keynote at Uberconf 2024 on Tuesday, July 16th. Definitely stop by if you're going to be there.
We released a bunch of new videos on our YouTube channel:
OpenRewrite v8.21.0 was released which includes bug fixes and some new recipes from some of our awesome community members.
The find and fix vulnerable dependencies recipe was updated to fix the version of transitive dependencies on top of direct dependencies.
We then welcomed Scott and Kyle as guest presenters to talk about JavaScript codemods. They are both senior developers here at Moderne, and they primarily specialize in frontend development. They:
Started off by explaining why we decided to invest into supporting JavaScript codemods in the platform.
Dove into some examples of codemods and discussed how we curated a list of important codemods.
Discussed how we created recipes for each of the ESLint stylistic plugin rules. On top of those recipes, we also made a larger ESLint recipe for styling recommended by ESLint.
Explained some of the challenges we faced when adding support for upgrading Angular versions.
Highlighted how you can use recipes to upgrade your JavaScript code to replace lodash objects and functions with native JavaScript.
Detailed what changes we'd still like to do and what things are still giving us trouble.
Walked through the Lint source code with ESLint recipe and explained how that recipe returns useful data tables and visualizations for tracking problems with your source code.
Explained how to create your own codemod recipes via a YAML file and provided an example for what this looks like.
Talked about what codemods they use most often in their day-to-day basis (e.g., improve regexes by making them shorter, consistent, and safer).
Refaster style recipes & Picnic's Error Print Support (March 12th, 2024)
Summary and related links
A quick run through of everything new in OpenRewrite & Moderne this week:
We published our Moderne Monthly newsletter to LinkedIn as well as online.
Be sure to subscribe to our monthly newsletter to stay up-to-date with all the latest Moderne news.
We added 86 more Refaster style recipes from Error Prone Support.
These recipes are designed to help you write better code and catch common mistakes, and now helpfully link to the docs to show more details of what changes to expect.
We split off a separate Rewrite recipe module for Apache projects, which you can also run through Moderne.
The reason behind this was to make recipes related to Apache projects easier to find, use and extend.
We created a new Rewrite recipe module for Node.js applications.
This provides the building blocks to change Node.js applications in a safe and automated way, starting with dependency management.
We also allow you to visualize dependency versions across your organization, to plan your next modernization steps.
Also new is a module for Struts applications, bringing folks from version 2.5 to 6.x.
This builds upon our recent addition of JSP parsing support, for more modernization capabilities.
Apache Wicket released their long awaited 10.0.0 release, with an automated migration using OpenRewrite.
We're excited to see the community using OpenRewrite to make their migrations easier and safer, and to have those recipes available on the day of the release.
Also available on the Moderne Platform, for easy use and extension.
For the main subject we looked at how the Picnic online supermarket uses both Error Prone Refaster rules and OpenRewrite recipes to enforce coding standards and catch common mistakes.
We saw how a team of five is able to support 150 Java developers in their day-to-day needs, through a high level of standardization and automation.
We went over some of their history, and how a past choice for TestNG was converted into JUnit 5 and AssertJ through a series of Refaster rules.
For large migrations, the platform team thoroughly prepares, automates and tests the changes, and then rolls them out in a controlled manner, ensuring no one is left behind.
Whenever there are multiple ways to do one thing, they build consensus and then automate the change, to ensure everyone is on the same page.
As for their Spring Boot 3 migration, they were able to migrate three million lines of code in just a few weeks through OpenRewrite recipes - with a high level of confidence in the changes.
The team at Moderne have since added support for Error Prone Refaster rules to the Moderne Platform, so you can use them in the same way as OpenRewrite recipes.
And we saw this in practice when Rick ran a Refaster rule to against the Apache Hive project's 21K files in just a couple seconds, and opened a PR with the changes.
If you're in Cologne during JCON Europe you can join Tim and Rick at their shared workshop on Transforming Code with OpenRewrite and Refaster.
Automate your code reviews with recipes (March 6th, 2024)
Summary and related links
We started off by covering all of the OpenRewrite and Moderne news for the past week:
We're excited to announce the Moderne DevCenter! OSS communities can use this to track progress on their migration goals (such as upgrading Java or Spring versions).
Some examples of this include the Spring DevCenter and the Apache Maven DevCenter.
Please note that, while we have the data about security vulnerabilities, we've chosen not to display it for OSS communities.
We created a new video showing you how to upgrade transitive dependencies in Gradle – which can help keep you safe from security vulnerabilities. We also wrote up a doc on this to go along with the video.
As a follow-up from last week, OpenRewrite has been officially added into the 2024.1 release of IntelliJ IDEA. You can find more details in their release blog.
We walked through the various benefits of this integration – such as autocompletion and being able to run recipes directly in IntelliJ.
We added support for parsing and visiting JSP files. We hope to use this to upgrade Struts 2.5 to 6.x.
We announced that we'll be returning to Spring IO this year. Come find us if you're there!
We then jumped over to the main topic: automating code reviews with OpenRewrite and the Moderne CLI.
We showed off a high-quality PR where we used this automation to ensure certain standards were followed (e.g., not including
System.out.println
in released code). One especially cool part of this is that the bot provides suggestions that the committer can simply accept without ever having to go back to the code themselves.You can find our OpenRewrite best practices recipe in the rewrite-recommendations repository or in the Moderne Platform.
We then dove into demonstrating how we did this:
The first step is running the receive-pr workflow – which then calls out to our shared workflow in our GitHub automations repository. The shared workflow runs the OpenRewrite recipes against the code in the PR.
The last step is commenting the changes back to the PR – which is what the comment-pr workflow does (which similarly calls out to our shared one).
If you want to learn more, check out the PR where we added this to the Langchain4j project.
We were largely inspired by this blog post from GitHub Security Lab. Definitely check that out if you're thinking about adding workflows to your projects so you can ensure you're doing so in a secure manner.
We then discussed when you may want to use the above automation (for highly homogenous organizations) vs. what options you have if your organization is more heterogenous and larger (in which case the Moderne CLI is a better fit as it abstracts away multiple languages and versions).
Data tables & search recipes (Feb 28, 2024)
Summary and related links
As normal, we started off by going over new things that occurred over the past week:
We added 95 Quarkus 3 & Camel migration recipes to the Moderne platform.
We demonstrated how you can use Refaster recipes to make your code clearer and more readable (e.g., optimizing your use of the Java time APIs)
In really exciting news, IntelliJ IDEA has added OpenRewrite to their early access branch. What this means is that you get native editor support for OpenRewrite YAML recipes and you can run recipes directly from your IDE. We'll have more information about this when it fully releases.
We added a bunch of reference recipes to the rewrite-recipe-starter repository so that it's easier to find a recipe to get started with. We also added Maven configuration for those users who prefer to use that over Gradle.
We wrote a blog post about using AI in refactoring. One big win was that code written in French would often times have misencoded special characters that would appear as question marks. We were able to use AI to determine what the appropriate character should be for valid French and replace the question mark with said valid character.
In a fun community win, Jonathan Leitschuh added the ability to find constant literal values which can greatly improve turning on feature flags. He also live streamed the development of this while answering questions from the community.
We then jumped into the main topic for the week: data tables and search recipes:
We demonstrated how you can use search recipes in Moderne to find usages of any type you care about (perhaps because you wanted to learn more about it or find some examples to copy).
We explained how Markers help call out individual elements and how they work in relation to search recipes.
We ran and talked through our Find and fix vulnerable dependencies recipe. We also walked through the data table it produced.
Using said data table, you can find transitive dependency vulnerabilities many levels deep – complete with CVE numbers and possible fix versions if they're available.
We then showed how you can produce your own class hierarchy and data table rows as well as how to verify the rows you produced.
Advanced recipe development: Scanning recipes (Feb 21, 2024)
Summary and related links
We started off by highlighting some of the big changes/additions since the last office hours:
There were some great community additions such as this new recipe on migrating away from Swagger to SpringDoc or OpenAPI.
We updated our Spring 3.2 blog to contain the latest Spring changes and also discuss Java 21 changes.
We then jumped into the core topic of Scanning Recipes. We walked through every part of developing a Scanning Recipe along with examples and tests.
Easy LST manipulation with Java Template (Feb 14, 2024)
Summary and related links
We highlighted a new blog post we put out about using AI search tool for finding recipes. We discussed why this AI search addition was important and what we were struggling with before that was added.
We discussed how we've started adding automated code reviews to our projects to ensure that code additions are consistent and align with our standards. This was done by running specific recipes against PRs.
We mentioned that we recently added JavaScript Codemods to our recipe marketplace. We'll discuss more about this in a different session, though.
We did a deeper dive into JavaTemplates and walked through how to debug and test them.
We have a detailed guide that walks you through how to use Java Templates to modify methods that you might find useful to follow along with.
We talked through what a Cursor is and how you might use them in your visitor.
Visitor pattern basics and Java Templating (Feb 7, 2024)
Summary and related links
We started off by going through the latest changes in OpenRewrite and encouraged users to make use of the rewrite-recipe-bom to ensure their versions are always aligned and up-to-date.
You can find all of the changes on the rewrite releases page. It's great to see so many people contributing!
We then highlighted some fun collaborations we've had. These include:
Tim being on the Quarkus podcast where he talked about how to use OpenRewrite to migration to the appropriate Java version to be ready for the latest Quarkus version.
Jonathan was on the DevProdEng Lowdown podcast where he talked about the collaboration between Gradle and Moderne and the new recipes added for Develocity configuration. They also discussed how Moderne has a scalable way to leverage Develocity Failure Analytics to find recipe misconfigurations.
We put out a blog recently about how our engineering team migrated to Spring Boot 3.2. Definitely check that out if you're interested in learning more about the entire process.
The core part of the office hours began with discussing Visitors and the visitor pattern. This is where the main part of each recipe lies. We do an in-depth walkthrough of some recipes and the visitors in them.
We talked a bit about Lossless Semantic Trees (LST) and provided some images that demonstrated what a Java program looks like when it's broken up into LST elements.
We briefly explained how to use JavaTemplates in recipes to construct complex LST elements.
There was a community question about what the difference is between an isomorphic visitor and a non-isomorphic visitor.
We discussed how you can use the rewrite-recipe-starter and the TreeVisitingPrinter to see what the LST looks like for the code you're interested in.
Moderne and OpenRewrite (Jan 24, 2024)
Summary and related links
We talked through how to get started with developing recipes. If you're looking to get started with that, check out the OpenRewrite authoring recipe docs.
You may also find it useful to start with a repository that let's you jump right into recipe development and provides some sample recipes to look at the code for. If so, check out the rewrite-recipe-starter we've made for this purpose.
We discussed the three different types of recipes you can create and what the pros/cons are of each.
We demonstrated how to create your own custom recipes quickly in Moderne without writing any code.
We talked through Refaster recipes and walked through creating and testing one.
If you want to make your own Refaster recipe, check out our Getting started with Refaster template recipes doc.
There was a community question about how to can contribute to OpenRewrite. To get started with that, check out our contributing doc where we walk through all the different ways you can help – many of which don't even involve writing code.
We walked through how to use Moderne to combine recipes like ChangeType and Add Gradle or Maven dependency into a larger migration recipe.
We explained what Preconditions are and how to use them.
You might also consider checking out the OpenRewrite docs on Preconditions.
Last updated