6 Things I Learned After 300 Hours of AI Coding
Useful tips borne from practical experience
“It’s mind-blowing, man. I don’t know the first thing about code, and with the help of Chat-GPT, I feel like I can build anything in just a few weeks.” I hung up the call with my friend, in awe of what I’d just heard and brimming with excitement to try to build my own apps and products using AI.
While my day-to-day responsibilities left me with just a few hours to dabble here and there, in April 2024, my workload lightened up considerably. With tons of free time, I decided to take a crack at building a web app, using AI as my partner in crime. I approached my project with gusto, hacking away into the wee hours, often resisting the urge to retire to bed until 3am or 4am. I consistently committed 4-6 hours daily to coding during that month. While my intensity cooled a bit after April, I have easily logged 300 hours of coding with the help of AI.
My conclusion? AI’s ability to write code is simultaneously one of the most mind-blowing things I have ever witnessed and also a far cry from the hype I was sold. In this article, I will share 6 practical tips I have learned about how best to leverage AI for your coding projects. I will also provide some pointers for building a project using AI. Anyone not interested in reading the full article can find the summary here (there’s also a video of me explaining my coding process below).
My coding background
Before diving in, I would be remiss not to share my unique coding background and experience. The idea of coding has always fascinated me. Still, I lacked the right resources and mentorship to excel at it for most of my life. I taught myself some PHP and SQL in college using an eBook I bought online, but it was rudimentary. I took a Java class in college, but the professor was so dry and uninspiring, and the course so poorly structured that I switched the class to pass/fail and barely squeaked over the line.
I got decent at reading Javascript through my digital advertising work. Still, it wasn’t until I did the coding bootcamp Hack Reactor in 2017 that I built a solid understanding of coding. While the program was only 3 months, it was another 3 months of highly intense prep, and by the end of the program, I’d logged over 1,000 hours of coding practice. I worked as an engineer at MightyHive for about 9 months, then started Stackmatix, my own growth agency, after which I didn’t touch code for another 6 years.
All this is to say: I am not a total rookie when it comes to coding. While highly rusty, I had a lot of muscle memory to work off of. I am generally a very technically inclined person and a savvy problem solver. However, I am also not a coding virtuoso. Left to my own devices, it would take me 6 months to build a project that a savvy software engineer could make in 3 to 4 weeks.
I’d estimate my experience level as similar to that of a gifted software engineer one year into a job or an average software engineer two years into a job. This makes me an interesting case study when it comes to leveraging AI: I know enough about how code works to get started and build, but I am not so wedded to the manual processes that I’m too proud to ask AI for help.
Finally, I have mainly used Github CoPilot and Chat-GPT as my AIs of choice for help with my coding projects. I have recently started testing Lovable to generate my first prototype and Cursor as my IDE. However, I am still relatively new and inexperienced with both tools, even though they are much more powerful (I made a video with a quick Lovable walkthrough below). Without further ado, the lessons I learned from my experience are as follows.
#1 - AI is like a brilliant intern
After becoming intimately acquainted with AI’s gifts and shortcomings, the best analogy I have developed is thinking of AI’s coding skills as that of a brilliant intern. Its brilliance is its ability to produce work of high quality at outrageous speed. However, the supreme confidence with which it shares results reminds me of a shy intern, obsessed with pleasing his boss but unsure of how to conduct himself, so he tries to always appear confident and in control. When nudged lightly or questioned, that same intern will apologize and, with the same nervous energy, produce another slightly flawed answer with exceptional confidence.
Often, the output I will get back from the AI has slight errors or inconsistencies. When you run the code and share the mistakes with the AI (e.g. “Chat-GPT, I’m seeing this error - what does it mean?”), You’ll get an answer that is equivalent to a digital facepalm. “Oh, yes, I see the issue now - we are missing these 3 lines. Here is the updated code that should now work smoothly.” You rerun the code, yet another issue pops up, and the cycle continues.
From my experience, this means that while AI can help get you 80% of the way there, you can’t shut off your brain entirely. You must be vigilant, evaluate the output, ask the right questions, and steer the AI toward the desired end goal. With the proper oversight, you can do amazing things. Still, you’ll never reach your destination if you’re asleep at the wheel.
#2 - Have a strong programming mentor or friend you can turn to for advice
It’s a true maxim that “we don’t know what we don’t know,” especially in programming. Given AI’s tendency to act like a brilliant intern, if you ask specific questions, it is unlikely to correct you, even if you are on the wrong track. In these moments, it is invaluable to have a strong programming friend or mentor who you can seek out for advice and help point you in the right direction.
I saw the value of having a strong programmer in my corner with my AI agent app. It only took me about 3 hours to build it from scratch and run it on my local computer (I describe that process here). But when I attempted to deploy it (i.e. make it accessible for others on the internet to use), it was a non-stop string of headaches that AI could not help me resolve.
The biggest hurdle was creating a variable that properly tracked my app’s progress. While it was being set correctly at certain moments, it also got reset randomly, which baffled me. My queries to AI to help troubleshoot it were fruitless, and I easily spent 10 hours banging my head against the wall in utter despair. Then, one weekday night, my roommate Kai came home early, and I asked for his input on what I was doing wrong. Given his background (Stanford computer science grad and engineer with 10+ years of experience), he had an immediate answer:
“Dude, you’re using Gunicorn as your production server, and it splits your app into 4 separate processes, so your agent is setting the variable properly on one thread, then a new thread is launched, and things start again from scratch. Global variables are useless in production; you need to use a database to track all key variables.”
That was it. The answer I had been so desperately seeking and that AI had been struggling to provide to me. When I queried Chat-GPT about this, my brilliant intern responded as expected, “Oh yes, that is correct, Gunicorn does have four threads, and a database is the best approach for variable storage.” Thanks for nothing…
I spent a few hours re-factoring my code to accommodate this new structure, tested it, and confirmed it worked flawlessly (you can try my app for yourself here). Case in point: AI is helpful, but a strong programming friend or mentor is as good or better.
#3 - Accelerate the feedback loop as much as possible
Coding with AI is a highly iterative process. You don’t give it a single prompt and get back a fully finished app; instead, each round of errors gets you one step closer to a fully fledged product. One thing I learned along the way is to accelerate the feedback loop to accelerate your learning process.
As a practical example, I built my AI agent app using the containerization platform Docker. The technical details of what Docker is are unimportant - all that matters is that each time you want to re-run your app using Docker, it can take 2-3 minutes to build and run it. I probably spent the first 10 hours of my coding project with this workflow, which meant that with the conservative estimate of 3 minutes per run, I could test out 20 code updates per hour (60 min / 3 min).
Eventually, I said, “There has to be a better way,” and learned about mounting Volumes in Docker. For the non-technical among us, Docker Volumes make it so that code updates can be run almost instantly instead of requiring the 2-3 minute build-time. Assuming it still takes 15 seconds to follow the new build and run steps, I can test 240 code updates per hour (60 min / 15 seconds). Thus, through a simple update, I have 20x’ed the speed at which I can test and learn, a mind-blowing difference in efficiency. Whether you’re using Docker or not, the more significant lesson is this: set up your project so that any update you make can be tested and learned from as quickly as possible.
#4 - Keep a running log for your project
I initially hoped that AI would spit out a finished app for me with a single prompt. When this proved false, I hoped AI would quickly help me overcome any bug to speed towards a finished product. Alas, that also proved illusory.
At times, AI was remarkably swift at correcting missteps, only needing a single round of feedback to get things humming. However, in other scenarios, AI would circle the drain for hours and hours on end, repeating the wrong answers it had given me previously. I’d go a whole week without feeling like I’d made any progress. It was then that I realized I needed a more intelligent approach to building and made two decisions:
1). I would log each change I made (an example of a rough log of mine can be found here) so I could track my progress and ensure I wasn’t making the same mistakes over and over.
2). Prior to making each change, I would do some thinking and generate a theory about what I expected this change to accomplish. I would then back-test the actual output against the theory to assess the misalignment.
Ultimately, I found this approach to be far superior. My initial strategy of copying and pasting code from Chat-GPT, getting an error, then asking Chat-GPT, “What does this mean?” worked well for easy issues. Still, when things got dicey, I realized I could easily stay stuck indefinitely if I didn’t approach things more thoughtfully.
Furthermore, for those periods in which I was consistently stuck, the ability to look back on my log and reflect on the things I’d learned and the solutions I’d ruled out was surprisingly reassuring. In addition, by tracking my mistakes and the solutions I found to overcome them, I had a great resource to refer back to in the future to ensure those learnings weren’t lost.
#5 - Start with a strong product requirements document (or at least know what you want)
There’s a well-known saying in engineering: garbage in, garbage out. In other words, the outputs you get in any process directly reflect your inputs. I have found this to be especially true with AI. The brilliant intern is a hesitant person’s pleasure, desperate for validation. Unless you push back on it, it won’t take the lead.
A good friend of mine, who is particularly brilliant at getting excellent output from AI, has shown me the power of a well-structured and thoughtfully constructed product requirement document. Frankly, this is an area I’m not yet particularly strong in (I tend to jump in and hack away until my product is built). Still, solid planning can save you a tremendous amount of time. The tech writer Peter Yang is someone who I think is particularly good at creating product requirement documents.
If the notion of a product requirement document sounds intimidating, I suggest that you at least have a clear idea of what you want and can articulate it clearly. With AI, sharing a few examples can be exceptionally powerful (e.g. “I want to mostly mimic Stripe’s homage design, but please change X, Y, and Z”). This is less about being a technical wiz and more about being thoughtful and methodical about determining what you want to build.
A simple example I had recently: I’m building a web scraper that pulls the HTTP requests from a homepage, looks for a search bar, simulates a page search, and then pulls an HTTP request from the final page (this app has a lot of practical utility for companies in the online privacy/compliance space). I explained to the AI that I wanted a web scraper that could capture HTTP requests, and it suggested I use a coding library called Playwright. I set this up, and it worked like a charm, but ran into issues when a site I was testing had a pop-up appear.
I spent 2-3 hours trying to find a solution to identify and close the pop-up, but then it hit me: I’m not just building for this one site; I need a flexible solution that can adapt to any site. I realized that I had been misguided, and an AI-based web scraper, which can mold itself for any page, was a far better fit. I also determined that had I initially provided clearer guidance to the AI (e.g. “This solution should work not just for one site, but be flexible enough to accommodate any site that is provided”), it would have given me the correct answer from the start.
Hopefully, zooming out from the technical nuances, my point is clear: thinking and planning from the get-go and communicating that to the AI will save you boatloads of time later.
#6 - Get some basic coding knowledge under your belt first
Blake Anderson, one of the most impressive people I have discovered in the indie hacker space, made millions of dollars building viral apps without any coding background. While Blake is an exceptional guy with tremendous grit, stories like his and hundreds of others circling social media give casual viewers the impression that you can quickly build production-grade apps by just slinging AI code at a wall and seeing what sticks.
But, as I hope I’ve clarified, it is nowhere near that simple. With Chat-GPT or Cursor helping you out, you might quickly get something basic off the ground. But, you will inevitably get stuck, find that AI will reach its limit in helping you, and you will need to do some serious study to push things forward.
When I was building my app in April 2024, I ran into so many issues with how my data was getting passed from my client side to the server side, especially around timing my data flows, that I had to spend a good 15-20 hours researching this subject and sharpening my skills in this area before I could grasp the root of the issue.
My hot take is this: if you don’t have the grit and determination to learn how to build and deploy a basic app without AI, you probably won’t have what it takes to build and deploy an app with AI. You will hit a snag, see your progress grind to a halt for 10+ hours, and eventually give up. Furthermore, theoretical coding concepts, such as the MVC Framework or how to set up your coding environment, are pretty abstract and tricky to understand.
My suggestion is to do a minimum of 20 hours of coding study before attempting to start building. This study should involve the basics of HTML, CSS, SQL, and React. The best resources around these subjects change constantly, but if you develop a basic understanding of each of these areas, you’ll be well-equipped to start building and can learn the rest through trial and error.
Conclusion
Building with AI is an incredible tool, but it’s far from a magic bullet. It’s like having a brilliant intern - fast, eager, confident, but occasionally flawed. Don’t rely on AI alone; find a strong mentor or community to back you up and accelerate your feedback loop to get results faster. Planning upfront and articulating your goals clearly will save you time and energy. At the same time, having foundational coding knowledge is essential for overcoming the inevitable roadblocks.
If you liked this content, please click the <3 button on Substack so I know which content to double down on.
TLDR Summary
This article shares six practical tips for leveraging AI in coding projects, based on my experience building apps with tools like ChatGPT and GitHub Copilot. While AI can accelerate coding, it’s far from perfect and requires thoughtful oversight, strong mentorship, and clear planning. The article emphasizes the importance of having some coding knowledge, using AI iteratively, and setting up projects in a way that maximizes learning and efficiency.
Key Sections and Insights
AI is Like a Brilliant Intern
AI can quickly generate code but often requires oversight to fix errors and inconsistencies.
It’s useful for getting 80% of the way but needs human input to reach the final goal.
Have a Strong Programming Mentor or Friend
AI can be helpful, but having a knowledgeable mentor can save time and resolve issues AI struggles with.
Real-world experience can fill in the gaps AI can’t address.
Accelerate the Feedback Loop
Iteration is key when coding with AI; speeding up feedback improves learning and progress.
Using efficient tools, like Docker Volumes, can drastically reduce testing times and increase productivity.
Keep a Running Log for Your Project
Tracking changes and documenting progress helps avoid repeating mistakes and ensures you stay on track.
This approach is more effective than relying solely on AI to remember previous steps.
Start with a Strong Product Requirements Document
Clear goals and a well-thought-out plan save time and guide AI toward the right solutions.
Properly articulating what you want from the beginning improves AI output and project outcomes.
It Helps to Have Some Coding Knowledge First
AI is a powerful tool, but without a basic understanding of coding, you’ll struggle when it encounters limitations.
Grit and foundational coding knowledge are essential for overcoming obstacles and completing projects.
Conclusion
Building with AI is a powerful tool that enhances productivity, but it’s not a magic bullet. Think of AI as a brilliant intern - it can assist greatly, but requires supervision, guidance, and a solid foundation in coding to achieve success. With the right mindset, support, and planning, AI can become an invaluable resource for your coding journey.
Work smarter not harder, right? I found this article so helpful and honestly cannot agree more with using AI to serve our productivity.