Code Remix

Background

Every week we host a live Code Remix session where we talk about the latest changes that are happening, answer community questions, and teach you more about specific topics.

You can find each of the Code Remix sessions below along with a summary of what was discussed and the key links you might find helpful.

Enlightening LSTs with traits (July 17th, 2024)

  • Coming soon

Automating Impact Analysis (July 10th, 2024)

Interview with Jente Sondervorst (July 3rd, 2024)

Overview of third party recipe libraries (June 26th, 2024)

An in-depth look at the Moderne CLI (June 19th, 2024)

  • Announcements for the week:

    • Releases:

      • We did a new release of all recipe modules, the Maven and Gradle plugins and the rewrite-recipe-bom.

      • We also did a new release of the Moderne IntelliJ plugin: v0.6.0

        • https://plugins.jetbrains.com/plugin/17565-moderne

      • As well as a new release of the Moderne CLI: v3.9.2

        • https://docs.moderne.io/user-documentation/moderne-cli/getting-started/cli-intro

    • Content:

      • We put out the June Moderne news letter, showing all that's new over the past month, and suggestion of what recipes to run over the summer.

      • Our new video on tech stack liquidity shows how you can consolidate from a variety of tech stacks to a single one, showcased through our AssertJ migration recipes.

      • A new blog takes you through our new recipe builder, showing both the functionality available to you, and the process we took to get there.

        • Use this new builder to quickly visualize existing recipes, and customize or extend them to your needs.

    • Events:

    • Promotion:

      • We continue our time limited promotion of the Moderne IntelliJ plugin & CLI. If you want to try it out, fill out our form.

      • You can now also book time with our experts Sam & Tim to get started quickly with the CLI on your projects.

  • From there we switched to the main topic for this week: An in-depth look at using the Moderne CLI.

    • We explained how the CLI is aimed at developers who want to run recipes across multiple repositories, and how it complements the Moderne IntelliJ plugin.

    • As a first step, you can use the CLI to discover git repositories, and execute a mod git pull ./ across all of them.

      • This is great for when you're getting back to a group of repositories after some time, pulling in all the latest changes at once.

    • From there, you might want to prepare to run recipes across all of those repositories, for which you can use the mod build ./ command.

      • This command will build all the repositories in the current directory, and create a .moderne directory with the LSTs of all the repositories.

      • Or alternatively, you can download the LSTs from artifactory, when those are built for you on a schedule.

    • Once you've acquired the LSTs, you'll want to find a recipe to execute, from your local copy of the recipe catalog.

      • There's a mod config recipes search option to help you find and select recipes to run.

      • A helpful shorthand is offered in the form of an active recipe, which you can also set from your IDE.

    • After selecting a recipe, you can run the recipe across all repositories, or a selection, using the mod run command.

      • --help and tab-completion are available to see how you can narrow down which repositories to run the recipe on.

      • When running a recipe, you'll notice changes are produced to a patch file, not directly written out to the repository.

      • This allows you to review the changes before committing them, and to run additional recipes in parallel.

    • Once you're satisfied with the changes, you can commit the results to a branch, and push them to the remote repository.

      • There are familiar git sub commands available to you for git operations across repositories.

      • The --last-recipe-run option allows you to only execute git commands against projects that were affected by the last recipe run.

    • Next we showed how the CLI can be used to gather insights across repositories, by running a recipe that produces a data table.

      • We ran a recipe to find and fix vulnerable dependencies, which analyzes all your direct and transitive dependencies.

      • The dependency versions you're using are cross-referenced against the GitHub Advisories database, to find the vulnerabilities that might affect you.

      • We explain how the recipe only confidently makes changes for .patch versions, following the Semantic versioning convention.

        • We're able to not just bump direct dependencies, but also transitive dependencies, with respective solutions for Maven and Gradle.

        • This is in contrast to other tools, which typically only handle direct dependencies, and lack insight into transitive dependencies.

        • That means a lot of vulnerabilities might otherwise go unnoticed, and unpatched.

      • Yet, even when the recipe does not produce code changes, we can still give you insight into additional vulnerabilities.

        • For that, we ran the suggested mod study to produce an Excel file showing exactly which dependencies are vulnerable.

        • This can be used to inform your team, and to prioritize your next steps based on severity, and mitigation options available to you.

      • Having these insights across repositories, and the tools to affect changes at scale, means you're in great shape when a new distruptive vulnerability might appear.

    • Next Sam demoed how you can run a recipe using the CLI, to then attach a debugger in IntelliJ.

      • This allows you to step through existing recipes, to better understand where they match, and what changes they produce.

      • We use this frequently to help harden our recipes, as there's always more variability in practice than you had thought of in advance in your unit tests.

    • Wrapping up we shared the form to get access to the Moderne CLI and IntelliJ plugin, such that you can make this part of your every day development.

Data flow analysis & recipe authoring best practices (June 12th, 2024)

Debugging recipes on real code (June 5th, 2024)

Moderne IDE plugin onboarding (May 29th, 2024)

OSS contributor conversation with DeShaun Carter (May 22nd, 2024)

OpenRewrite parsers (May 15th, 2024)

  • Announcements for the week:

  • OpenRewrite parsers:

    • There are two types of parsers in OpenRewrite: parsers we build around an existing language compiler and there are parsers that we build around an ANTLR grammar.

    • We began by discussing the Java parser (which is based on an existing language compiler) and how we interact with the Java compiler to map objects to our Lossless Semantic Tree.

    • We then stepped through the ReloadableJava17Parser and explained some key parts of the parser.

    • While stepping through the parser, we also jumped into the ReloadableJava17ParserVisitor and talked through why it's a bit different from the traditional visitor you may be used to.

    • A lot of what goes on in the parser is taking the Java compiler's abstract syntax tree (AST) and the raw textual representation of the source file, and traversing through both in tandem to match up white space from the raw source file to the structured objects the Java compiler gives us. That then lets us produce the LST that has both of those pieces married together.

    • To help with stepping through the code and figuring out where in the String we're currently reading from, we use a cursor object.

    • Next up was talking about type attribution. OpenRewrite has its own representation of Java types. To get from a Java representation of a type to the OpenRewrite representation of a type, we've created the ReloadableJava17TypeMapping.

    • For testing types, we've created a JavaTypeGoat class. The purpose of this class is to test all of the strange things that go on with typing and make sure we have a sort of all-encompassing class that can test the edge-cases or things you wouldn't normally see.

    • We then jumped over to talking about parsers that are based on ANTLR grammars and what the pros/cons of that are compared to a parser based on an existing compiler.

    • As part of demonstrating these types of parsers, we took a look at the JSON ANTLR files and demonstrated how you can test input to see if it matches said grammar.

    • We then looked over what all is generated by ANTLR and how we translate the generated code to OpenRewrite classes. Similar to before, we use cursors to keep everything aligned as we ensure white space is included.

    • We wrapped up the talk by taking a brief look at some of the other parsers and compilers and discussing what the difficulties were with each – including a hint at the upcoming C# parser.

Weeding your microservice landscape (May 8th, 2024)

Contributing to OSS through Moderne (May 1st, 2024)

OSS contributor conversation with Shannon Pamperl (April 24th, 2024)

  • Announcements for the week:

    • Releases:

      • We've done a new full release of rewrite and the rewrite-recipe-bom. For a full list of changes, check out our changelog over on the OpenRewrite docs. Special shout out to the folks over at IBM who contributed a ton of changes on the rewrite-migrate-java project.

    • Events:

      • Tim will be at JCON Europe from May 13th - 16th. He will be giving a talk on Transforming Code with OpenRewrite and Refaster and Weeding your Micro Service Landscape. He has a limited amount of free tickets to give away if you'd like to attend and are going to be in Germany at that time. Please reach out to him to learn more.

      • Tim and Jonathan will be at Spring I/O from May 30th - 31st in Barcelona. They'll be giving a talk on automated software refactoring with OpenRewrite and generative AI.

      • Jonathan and Olga will be at UberConf in Denver on July 16th - 19th. They will be giving two workshops plus a keynote presentation!

      • Tim gave a short summary of what happened at Devnexus (one of the conferences he recently attended). We were especially proud of the fact that many other speakers mentioned us and OpenRewrite in their presentations. Our AI talk was also packed with people wanting to learn more – which was exciting. We also learned some fascinating things from the community such as the fact that some places are still looking to update from Spring 1 and 2. If you're one of those places, make sure to check out our Spring Boot 3.x recipes that can help!

    • Content:

    • Press/Mentions:

      • OpenRewrite has joined the Commonhaus Foundation. The Commonhaus Foundation is a new model for established open source libraries and frameworks seeing a neutral home. The non-profit's mission is to ensure a place where open source projects can continue to thrive for future generations. This shows our commitment to OpenRewrite remaining safe and open source even if something were to happen to Moderne. For more information, check out the press release.

      • Netflix featured us in their presentation at Devnexus where they were talking about modernizing and cleaning up their code.

  • We then introduced Shannon Pamperl. He's one of the oldest and most prolific contributors to OpenRewrite. We then asked had a lengthy conversation with him on his history with OpenRewrite. Some of the questions that Shannon answered include:

    • How did you get started with OpenRewrite?

      • Knew Jonathan from a previous job and always respected his work. Got a message from Jonathan and began looking into the project right away.

    • What were some of the problems you used OpenRewrite to solve?

      • Wanted to use it with Gradle. Helped kick off discussions that led to rewrite-groovy and rewrite-gradle.

    • What are some of the challenges you had when learning to use and create recipes for OpenRewrite?

    • Were there any particularly challenging contributions you made?

    • You've been working on Gradle Kotlin support - how has that been going?

      • For both of these, difficulty has been with type attribution.

      • Happy to have help with this project. If you want to, consider contributing to rewrite-gradle-experimental.

    • We've found that Shelter engineers are extremely competent and have technical depth in a variety of fields. Is there something that Shelter does right that helps nurture great engineering talent?

      • Shelter stands by their engineers. They push you to do your best while also understanding that people don't know everything. When they don't know something, they'll work together to figure it out without judgement. Every individual person is taught and levels up over time – which compounds when there's a team of people like that.

    • If you could make a wish right now on one thing OpenRewrite or Moderne would do that it doesn't right now, what would that be?

      • More parsers for more languages. TSX would be super valuable. TypeScript/React/HTML compounded together can be tricky to handle.

    • Are there any tips/tricks you have that newer recipe authors would benefit from?

      • Focus on doing one thing really well. When making a recipe, make it simple and focused and ensure that it completely solves the core issue. Then, you can combine multiple of these recipes together to make something more substantial – and you can be confident in the individual pieces.

    • Are there any libraries or common platforms that you wish there was a recipe for?

      • Most libraries that he uses already have recipes for them, but they lack features in them. When he finds issues that are important, he tries to contribute back recipes for them.

    • If someone wanted to get started with OpenRewrite, what would you recommend?

      • Start by coming at OpenRewrite with a problem that you want to solve, so you have some context. Then, join the OpenRewrite community Slack and ask questions in there. He'd also recommend looking at RewriteTest and rewriteRun to get a better idea of the flow of recipes.

      • (We'd also recommend checking out our documentation where we've created numerous tutorials and references to help people get started. If you run into any issues or wish for something to exist that doesn't, please let us know!)

    • Could you tell us about the challenges with using feature flags that motivated you to make contributions to LaunchDarkly recipes?

      • At big enterprise companies, it can be very hard to keep up with the open source community as it moves so fast. Feature flags may get added and then forgotten about. Having recipes to find and remove these has been super helpful.

    • Are you using LaunchDarkly recipes to get rid of feature flags at Shelter?

      • Yes. It's much nicer to use recipes than manually doing it across systems.

    • Is there anything cool that we should have asked that we missed?

      • Shannon was curious what's going on with cycles in relation to recipes. Sam dove into some of the issues that we've run into with trying to eliminate cycles.

    • Is there anything missing from the recipe execution lifecycle?

      • Unsure. Biggest problem right now is ScanningRecipes behaviour when there are multiple of them trying to interact with a single top-level LST element. Mentioned that the individual elements just works but that the combination causes problems.

    • Is there a way to make it easier to compose ScanningRecipes together?

      • Probably. Not sure on exact details. If you (the person reading this) have any ideas, definitely let us know!

Recipe authoring tips (April 17th, 2024)

  • Announcements for the week:

  • We then jumped into the main topic for the week – tips and tricks for recipe development:

    • We began by looking at the Gradle Test use JUnit Jupiter recipe, which, on paper, seems like it should a relatively straightforward recipe. The core idea behind it is we want to useJUnitPlatform() to the Test task configuration for Gradle projects. It's only a line or two of changes - but there are many cases to consider. For instance, the existing project may not have any test tasks set up, in which case you would need to add that along with the changes. Other projects may already have a test task set up, and they are stylized in a certain way, in which case you would want to match their existing style and add the configuration alongside of it.

    • We then went through a pattern we use when developing recipes like this. We start off by ensuring we don't make changes when they're inapplicable or when we can't be confident enough they would be correct. We then try and find cases where no change is required as the change has already been made. This helps ensure the recipe isn't making any unnecessary changing and also helps ensure we aren't wasting processing time or power when it's not needed. After that, there are several layers of attempting to update existing configuration. Only after everything else has been handled, do we attempt to add new configuration.

      • The core idea with this pattern is to start off with the smallest pieces and work your way up to the most complex. This not only helps improve the speed of the recipe results, but it also makes it easier to create tests and gain confidence as you go.

    • Another important note with this recipe is that we created multiple visitors that are used throughout the base part of the recipe. This helps improve readability as the core flow through the recipe is greatly simplified. This also helps improve testability as you can test more focused visitors or paths.

    • We then dove into the individual visitors to help explain what's happening and why. One notable caveat with Gradle visitors is that we can't be as confident in types – so we have to perform some additional checks to ensure that we are parsing/modifying the correct thing.

    • The next topic was talking about the difference between a traditional, imperative recipe and a scanning recipe. The core idea being that regular recipes can not be used if you need to take information from one file to modify something in another file as they only look at each file once in a non-configurable order. Scanning recipes, on the other hand, get access to different phases of the recipe lifecycle (such as the scanning phase which comes before the edit phase and allows you to look through all the files, but not make any changes to them).

    • We then dove into recipes that are composed of multiple scanning recipes – such as the AddDependency recipe. We talked through some things you'll need to think about and be aware of when writing your own scanning recipes – such as ensuring scanners are created for each recipe you want to use.

AI integrations at Moderne (April 4th, 2024)

  • This week we welcome Justine - our AI/machine learning research engineer!

  • Announcements for the week:

  • AI topic:

    • We started off by talking through search in Moderne. Prior to AI search being added, it was very hard to discover recipes, typos would result in no recipes being found, and words that were tangentially related did not provide results. Adding AI search helped fix all of these issues.

    • We then provided some more context into how AI search works and how you might get started with your own. Check out Hugging Face to look through various models - which includes the ones we used.

    • We talked through common questions about these models such as how they work, how they're trained, what type of system you need, etc.

    • If you want to learn more details about AI at Moderne, Justine also wrote a lovely blog post on fast, secure, and cost-effective AI searching.

    • Next up was diving into various types of visualizations and how they can be used to discover important elements in your code. For instance, we highlighted a visualization that shows the different languages that are in use in your comments - and one that clusters method names together.

    • We provided an example of how we used AI to help one of our customers who had mis-encoded French comments throughout their code base. This made it a struggle to automate some key things they wanted to accomplish such as documentation generation.

    • We concluded by talking through some upcoming AI additions to Moderne. One of these is a way of using AI to search through code. For instance, if you had a general idea of what you wanted to search for but weren't quite sure the exact thing to search for, you could use this to find related code.

    • Another upcoming change is using AI to make suggestions of recipes to run on your code base to make it more secure. For instance, if it saw you were using an improper random number generator, it might recommend running a recipe to change it to use a secure one. These recipe suggestions will appear in the DevCenter for the organization.

Dependency management recipes (March 25th, 2024)

  • Only a few announcements this week since it's so close to the last office hours:

  • The main topic was about dependency management:

    • We talked about how to gain insight into your dependencies. For instance, maybe you're curious about if all of your repositories are using the same version of Jackson across all of their modules.

      • You can use the visualizations or data tables produced by this recipe to get detailed information about that.

    • Once you've dug into what dependencies you're using across your repositories, you may want to upgrade your dependencies to be on the same version.

      • This is also a really useful recipe to use when writing migration recipes such as newer versions of Spring.

    • The code for both of the above recipes can be found in the rewrite-java-dependencies repository.

    • Another common thing developers need to do is update their Gradle wrapper as new versions of Gradle are released. We demonstrated how you can use the Update Gradle wrapper recipe to update this across all of your repositories very quickly.

    • We highlighted some recipes that were enhanced recently to help with dependencies. For instance, in the change Maven parent recipe, we've added support so that properties that used to exist in the parent pom (but don't anymore) can be brought down into the child pom so that the project can keep compiling until you can upgrade it.

    • To go along with the above recipe change, we also updated the Remove redundant explicit dependencies recipe so that you can remove dependencies that exist in both your current pom and the parent pom – with options to let you decide if you only want to do that when the versions match exactly or not.

    • As part of investigating dependencies, it's a good idea to check for and possibly fix vulnerable dependencies.

    • We concluded by highlighting the dependency tree generated by the dependency report recipe. This is a searchable tree that helps you answer the question of, "What do I have that includes this dependency?" or "What uses this specific version of this dependency?"

Running JavaScript codemods at scale (March 20th, 2024)

Refaster style recipes & Picnic's Error Print Support (March 12th, 2024)

  • A quick run through of everything new in OpenRewrite & Moderne this week:

  • For the main subject we looked at how the Picnic online supermarket uses both Error Prone Refaster rules and OpenRewrite recipes to enforce coding standards and catch common mistakes.

    • We saw how a team of five is able to support 150 Java developers in their day-to-day needs, through a high level of standardization and automation.

    • We went over some of their history, and how a past choice for TestNG was converted into JUnit 5 and AssertJ through a series of Refaster rules.

    • For large migrations, the platform team thoroughly prepares, automates and tests the changes, and then rolls them out in a controlled manner, ensuring no one is left behind.

    • Whenever there are multiple ways to do one thing, they build consensus and then automate the change, to ensure everyone is on the same page.

    • As for their Spring Boot 3 migration, they were able to migrate three million lines of code in just a few weeks through OpenRewrite recipes - with a high level of confidence in the changes.

    • The team at Moderne have since added support for Error Prone Refaster rules to the Moderne Platform, so you can use them in the same way as OpenRewrite recipes.

    • And we saw this in practice when Rick ran a Refaster rule to against the Apache Hive project's 21K files in just a couple seconds, and opened a PR with the changes.

  • If you're in Cologne during JCON Europe you can join Tim and Rick at their shared workshop on Transforming Code with OpenRewrite and Refaster.

Automate your code reviews with recipes (March 6th, 2024)

Data tables & search recipes (Feb 28, 2024)

Advanced recipe development: Scanning recipes (Feb 21, 2024)

Easy LST manipulation with Java Template (Feb 14, 2024)

Visitor pattern basics and Java Templating (Feb 7, 2024)

Moderne and OpenRewrite (Jan 24, 2024)

Last updated