Using AI to Script Code Modifications
By Juan Caicedo
Introduction
Have you ever been faced with a long and tedious code migration that stood in the way of you completing an exciting project? What if you knew there was a tool that could do that task perfectly, but learning to use the tool was a bigger investment than you were willing to put in?
That's the situation I faced setting up internationalization (i18n) for Calm Health, and I found that AI was able to free my focus from learning a novel API to instead tweaking a solution to do what I needed. In this post, I'll show how I used targeted examples to prompt Glean AI to write code transformation scripts for me.
A Note on Glean AI
For this work, I used Glean’s AI chat feature given they are an approved vendor that meets Calm's security and privacy standards. This same strategy would work similarly with ChatGPT or any other generative AI tool that is able to work with code.
The mission
One of our big goals in 2024 was to support the Calm Health web application in Spanish. At the onset of the project, our team had the foresight to externalize all of our strings through an i18n library, but as the project progressed, we found that we needed to migrate from one i18n library to another. We had used next-i18next, whereas other web applications at Calm use react-intl (format.js). We wanted to consolidate, so as part of this project we would migrate all our code using next-i18n to react-intl.
Moving to react-intl meant changing all our import statements from one library to another. We had built two different wrappers to allow us to interface with the library, but both of these would need to change.
# Before
import { useTranslation } from '@rhg/libs/i18n';
import useLazyTranslation from '@rhg/hooks/use-lazy-translation';
# After
import { useIntl } from 'react-intl';Another complication was that next-i18n and react-intl store translation strings in different formats, which then need to be referenced in different ways.
In next-i18next, the strings to be translated are stored in nested json files, each of which contains a json object with reference keys and values defining how the string should appear in the default language (english). The library compiles all these files together, so at runtime you can reference them through a string which specifies which file they're stored in and then the key referencing the nested location of the string.
/* common.json */
{
"today": "Today",
"greeting": {
"morning": "Good morning!"
}
}
/* Component */
function MyComponent() {
const { t } = useTranslation();
// This is another variation
// const { t } = useLazyTranslation();
return <div>{t('common:greeting.morning')}</div>
}In react-intl, translation strings are referenced as JS objects. They are the result of calling a function. When using the translation function in a component, you need to reference a property pointing to the translation string.
/* commonMessages.ts */
import { defineMessages } from 'react-intl';
export default defineMessages({
"greeting.morning": {
id: "common.greeting.morning",
defaultMessage: "Good morning!",
description: "A greeting at the start of the day"
},
"today": {
id: "common.today",
defaultMessage: "Today",
description: "The current day"
}
});
/* Component */
import commonMessages from './commonMessages.ts'
function MyComponent() {
const { formatMessage } = useIntl();
return <div>{formatMessage(commonMessages['common.greeting.morning']}</div>
}The challenges of a code migration
We had over a thousand strings already registered in the old library. I tried to manually migrate some of these function calls over, but migrating about 3% of our strings took me a few hours. That meant that what I had to look forward to was multiple days (if not weeks) of boring and tedious string migrations.
Luckily, I'm not the first person to need to do this type of large code base modifications. There are some well-established tools for doing things like this. The one we've used previously at Calm is jscodeshift.
However, rightfully or not, I perceived this tool to have a big learning curve. Though the API is not particularly complex, learning to understand and navigate code is. After your code is ingested by the library, it is understood through an Abstract Syntax Tree, with its nodes needing to be navigated and mutated into the structure you want.
I didn't know where to start in understanding this new domain, and to be honest, I didn't really want to learn.
Enter AI
The thing about these code modification scripts is that they are very self-contained and they are easy to describe. That is the type of thing that AI is very capable of handling. I thought of trying writing my code modification script through Glean AI.
First, I had Glean write some scripts for migrating my translation files from the next-i18n format to react-intl. This helped me test out the concept of generating transformer scripts, and to see how the tool would work before modifying all of my src files.
Afterward, I had it write a more involved script for changing all the function calls from the old library over to the new library. When the old library references a file in the string path, I now needed to import the file where that translation was referenced and to refer to the correct property.
In both cases, I listed out all the steps that I needed my transformer to cover, and I included examples for each step. To my delight, the output transformer from Glean worked almost exactly like I needed it to! This saved me a huge amount of work in the end.
For brevity, I won't discuss the prompting in depth here, but I’ll attach my full transcripts in case you are looking for more guidance.
Shortcomings
There were a few significant flaws with the solution that it provided. I found these by running the transformer on a file and looking at what came out.
One issue was that it wasn’t able to change the default import of useLazyTranslation over to the destructured import of { useIntl }. I could have tried to craft another prompt to get a better solution, but I decided it wasn't really worth it. I was able to edit my transformer to insert that destructuring as if it was a function name, which worked fine. There might have been a more technically correct implementation, but I was just focused on getting the result I cared about.
Another problem was that the order of operations that Glean generated was incorrect. One step looked for all cases where a file name was used in the translation path (common:) and added an import for them, but the previous step changed all paths to the new format needed by react-intl (common:greeting.morning -> common.greening.morning), which meant none of those cases would be found for the import step. Again, I modified the script myself and reran it to see it fix the problem.
Takeaways
In this case, I found that leveraging AI was useful to get the wheels moving in the right direction. This technology isn't yet at a place where it can do a task like this reliably without intervention. However, to be useful, it doesn't need to be. By coming up with a passable solution that could use the right APIs to do something in the general sense of what I needed, Glean made it so that I could focus more of my energy into identifying and remedying the fallbacks of what it had provided. This job felt much easier to me, and it was more fun than trying to learn the solution myself.
I also learned that Glean's AI works very well for code when you can provide it with examples. Describing functionality can be difficult, but given an example of what the solution needed to do, Glean was able to figure out a way to achieve it.
Like all information coming out of LLMs, we shouldn't blindly trust it. Being able to run the codemod and inspecting the results was a crucial step for shaping the actual solution I needed. If I were taking on a more complex code modification, I would put the behaviors I want into unit tests, showing the code before and after. That would give me a fast automated way to run a transform and make sure it does all the things that I want, which would enable me to iterate on the solution more quickly. In fact, those unit tests could serve as the actual prompt to give Glean.
Conclusion
This is one of the places within programming where AI has the most to offer right now. When working with a novel API, Glean can provide enough context in the new domain so that I can shape it into the solution I want.
Now that I've been able to see jscodeshift in action and how it achieves something I want, it would be easier for me to go back and learn some of the concepts of how it works at a deeper level. But the real value is that I didn't need to. I was able to leverage the tool and achieve my goals with it before needing to invest in that learning.
Appendix (Glean transcripts)
These are abbreviated versions of my conversation with Glean AI that illustrate how to generate the codemod scripts. Please note that they may not work right out of the box and will still require some debugging.
