Senior Developer Skills in the AI Age

The more I browse through this, the more I agree. I feel like one could delete almost all comments from that project without losing any information – which means, at least the variable naming is (probably?) sensible. Then again, I don't know the application domain.

Also…

  def _save_current_date_time(current_date_time_file: str, current_date_time: str) -> None:
    with Path(current_date_time_file).open("w") as f:
      f.write(current_date_time)

The scary thing is: I have seen professional human developers write worse code.

Ok - not wrong at all. Now take that feedback and put it in a prompt back to the LLM.

They’re very good at honing bad code into good code with good feedback. And when you can describe good code faster than you can write it - for instance it uses a library you’re not intimately familiar with - this kind of coding can be enormously productive.

My current favourite LLM wankery example is this beauty: https://blog.fahadusman.com/proxmox-replacing-failed-drive-i...

Note how it has invented the faster parameter for the zpool command. It is possible that the blog writer hallucinated a faster parameter themselves without needing a LLM - who knows.

I think all developers should add a faster parameter to all commands to make them run faster. Perhaps a LLM could create the faster code.

I predict an increase of man page reading, and better quality documentation at authoritative sources. We will also improve our skills at finding auth sources of docs. My uBlacklist is getting quite long.

In my opinion this isn't even too relevant. I am no python expert but I believe defining a logger at the top for the average one file python script is perfectly adequate or even very sensible in many scenarios. Depends on what you expect the code to do. Ok, the file is named utils.py...

Worse by far is still the ability of AI to really integrate different problems and combine them into a solution. And it also seems to depend on the language. In my opinion especially Python and JS results are often very mixxed while other languages with presumably a smaller training set might even fare better. JS seems often fine with asynchronous operation like that file check however.

Perhaps really vetting a training set would improve AIs, but it would be quite work intensive to build something like that. That would require a lot of senior devs, which is hard to come by. And then they need to agree on code quality, which might be impossible.

I "love" this part:

  def ensure_dir_exists(path: str) -> None:
    """
    Ensure a directory exists.

    Args:
        path: Directory path
    """

    # Ensure the directory exists and is writable
    ensure_dir_exists(work_dir)

    work_path = Path(work_dir)
    if not work_path.exists() or not os.access(work_dir, os.W_OK):

This might not matter here because it's a small, trivial example, but if you have 10, 50, 100, 500 developers working on a codebase, and they're all thoughtlessly slinging code like this in, you're going to have a dumpster fire soon enough.

I honestly think "vibe coding" is the best use case for AI coding, because at least then you're fully aware the code is throwaway shit and don't pretend otherwise.

edit: and actually looking deeper, `ensure_dir_exists` actually makes the directory, except it's already been made before the function is called so... sigh. Code reviews are going to be pretty tedious in the coming years, aren't they?

Not all code needs to be written at a high level of quality. A good deal of code just needs to work. Shell scripts, one-offs, linter rules, etc.

100%!

But the alternative would be the tool doesn't get built because the author doesn't know enough Python to even produce crappy code, or doesn't have the money to hire an awesome Python coder to do that for them.

Also somewhat strangely, I've found Python output has remained bad, especially for me with dataframe tasks/data analysis. For remembering matplotlib syntax I still find most of them pretty good, but for handling datagframes, very bad and extremely counter productive.

Saying that, for typed languages like TypeScript and C#, they have gotten very good. I suspect this might be related to the semantic information can be found in typed languages, and hard to follow unstructured blobs like dataframes, and there for, not well repeated by LLMs.

Perhaps that partly because 90% of the training data used to teach LLMs to code is made by junior engineers?

> This is especially noteworthy because I don’t actually know Python.

> However, my broad understanding of software architecture, engineering best practices, system operations, and what makes for excellent software projects made this development process remarkably smooth.

If the seniors are going to write this sort of Python code and then talk about how knowledge and experience made it smooth or whatever, might as well hire a junior and let them learn through trials and tribulations.

As an actually unseasoned Python developer, would you be so kind as to explain why the problems you see are problems and their alternatives? Particularly the first two you note.

Here's a rl example from today:

I asked $random_llm to give me code to recursively scan a directory and give me a list of file names relative to the top directory scanned and their sizes.

It gave me working code. On my test data directory it needed ... 6.8 seconds.

After 5 min of eliminating obvious inefficiencies the new code needed ... 1.4 seconds. And i didn't even read the docs for the used functions yet, just changed what seemed to generate too many filesystem calls for each file.

Thanks for looking into it.

While I would have hoped for a better result, I'm not surprised. In this particular case, I really didn't care about the code at all; I cared about the end result at runtime, that is, can I create a working, stable solution that solves my problem, in a tech stack I'm not familiar with?

(While still taking care of well-structured requirements and guard rails — not to guarantee a specific level of code quality per se, but to ensure that the AI works towards my goals without the need to intervene as much as possible).

I will spin up another session where I ask it to improve the implementation, and report back.

>to a raciness in load_json where it's checked for file existence with an if and then carrying on as if the file is certainly there...

Explain the issue with load_json to me more. From my reading it checks if the file exists, then raises an error if it does not. How is that carrying on as if the file is certainly there?

Makes sense given that so much of the training data for so many of these tools are trained on hello world examples where this kind of configuration is okay. Not like this will matter in a world where there are no juniors to replace aged-out seniors because AI was "good enough"...

> to a raciness in load_json where it's checked for file existence with an if and then carrying on as if the file is certainly there...

It's not a race. It's just redundant. If the file does not exist at the time you actually try to access it you get the same error with slightly better error message.

How do you properly configure a logger in application like that?

Doesn’t load_json throw if the file doesn’t exist?

I disagree, I think it's absolutely astounding that they've gotten this good in such a short time, and I think we'll get better models in the near future.

By the way, prompting models properly helps a lot for generating good code. They get lazy if you don't explicitly ask for well-written code (or put that in the system prompt).

It also helps immensely to have two contexts, one that generates the code and one that reviews it (and has a different system prompt).

Yup this tracks with what I have seen as well. Most devs who use this daily are usually junior devs or javascript devs who both write sloppy questionable code.

wrap_long_lines shares those characteristics:

https://github.com/dx-tooling/platform-problem-monitoring-co...

Where things are placed in the project seems rather ad hoc too. Put everything in the same place kind of architecture. A better strategy might be to separate out the I and the O of IO. Maybe someone wants SMS or group chat notifications later on, instead of shifting the numbers in filenames step11_ onwards one could then add a directory in the O part and hook it into an actual application core.

Thanks for doing the footwork. These TED talk blog posts always stink of phony-baloney nonsense.

I'm the complete opposite. After being burnt out and feeling an almost physical repulsion to starting anything new, using AI has renewed my passion. I've almost finished a side project I started 4 weeks ago and it's been awesome. Used AI from the beginning for a Desktop app with a framework I'd never heard of before and the learning curve is almost non-existent. To be able to get the boring things done in minutes is amazing.

I can echo your sentiment. Art is the manifestation of creativity, and to create any good art you need to train in whatever medium you choose. For the decade I've been a professional programmer, I've always argued that writing code was a creative job.

It's been depressing to listen to people pretend that LLM generated code is "the same thing". To trivialize the thoughtful lessons one has learned honing their craft. It's the same reason the Studio Ghilbi AI image trend gives me the ick.

I've become so much more productive in my hobby programming projects, because it lets me skip over the tedious parts that suck the life out of me (boilerplate, figuring out library dependencies, etc), and I can focus on the parts that are actually interesting like experimenting with different solutions, iterating and learning new things.

It also lets me explore other libraries and languages without having to invest too much time and effort before knowing if it's right for what I want to do. I know that if I want to continue in a different direction, I haven't wasted tons of time/effort and getting started in the new direction will be much less effortful and time consuming.

I know what you mean. It takes the satisfaction out of a certain class of problems knowing that you can just generate the solution.

On the other hand, most tasks aren't fun nor satisfying and frankly they are a waste of time, like realizing you're about to spend the afternoon recredentializing in some aspect of Webpack/Gradle/BouncyCastle/Next.js/Combine/x/y/z just to solve one minor issue. And it's pure bliss when the LLM knows the solution.

I think the best antidote to the upset in your comment is to build bigger and more difficult things. Save your expertise for the stuff that could actually use your expertise rather than getting stuck wasting it on pure time burn like we had to in the past.

Try being a drummer!

I'm mixed on the "craft" aspect of it. I don't hate coding, but there's definitely times where it feels like it's just some time plumbing. I don't get satisfaction out of that kind of stuff.

I'm pragmatic and simply don't trust current LLM's to do much in my domain. All that tribal knowledge is kept under lock and key at studios, so good luck scraping the net to find more than the very basic samples of how to do something. I've spent well over a decade doing that myself; the advanced (and even a lot of intermediate) information is slim and mostly behind paywalls or books.

Fascinating. it's gone the other way for me. because I can now whip up a serious contender to any SaaS business in a week, it's made everything more fun, not less.

I would love to be a junior right now. I would just hate becoming a senior, after having been a junior right now.

Well, whats the difference between Vibe Coding in 5 - 10 years vs coding in C 5 - 10 years after compilers came out?

It’s also interesting to see how quickly the greenfield progress rate slows down as the projects grow.

I skimmed the vibecoding subreddits for a while. It was common to see frustrations about how coding tools (Cursor, Copilot, etc) were great last month but terrible now. The pattern repeats every month, though. When you look closer it’s usually people who were thrilled when their projects were small but are now frustrated when they’re bigger.

Oh, I find almost the exact opposite.

On Greenfield projects there's simply too many options for it to pursue. It will take one approach in one place then switch to another.

On a brownfield project, you can give it some reference code and tell it about places to look for patterns and it will understand them.

And greenfield turns into brownfield pretty quickly.

I find that feeding in a bunch of context can help you refactor, add tests to a low coverage application pretty quickly, etc in brownfield apps.

My experience on brownfield projects is the opposite.

Right, but AI could change the ratio of greenfield vs brownfield then (« I’ll be faster if I rewrite this part from scratch »)

I think there's a similar analogy here for products in the AI era.

Bolting AI onto existing products probably doesn't make sense. AI is going to produce an entirely new set of products with AI-first creation modalities.

You don't need AI in Photoshop / Gimp / Krita to manipulate images. You need a brand new AI-first creation tool that uses your mouse inputs like magic to create images. Image creation looks nothing like it did in the past.

You don't need Figma to design a webpage. You need an AI-first tool that creates the output - Lovable, V0, etc. are becoming that.

You don't need AI in your IDE. Your IDE needs to be built around AI. And perhaps eventually even programming languages and libraries themselves need AI annotations or ASTs.

You don't need AI in Docs / Gmail / Sheets. You're going to be creating documents from scratch (maybe pasting things in). "My presentation has these ideas, figures, and facts" is much different than creating and editing the structure from scratch.

There is so much new stuff to build, and the old tools are all going to die.

I'd be shocked if anyone is using Gimp, Blender, Photoshop, Premiere, PowerPoint, etc. in ten years. These are all going to be reinvented. The only way these products themselves survive is if they undergo tectonic shifts in development and an eventual complete rewrite.

I wonder how much time it would take with some samples from GitHub, and various documentation about python laying around (languages, cheatsheet, libraries)...

[flagged]

> I have noticed many more errors are detected in AI-generated Rust code than in any other language.

Is that because the Rust compiler is just a very strong guardrail? Sounds like it could work well for Swift too. If only xcodebuild were less of a pain for big projects.

> Building a personal brand is never more important

the low-hanging fruit is to create content/apps to help developers create their personal brands through content/apps.

Waterfall has always been the best model as long as specs are frozen, which is never the case.

Great observations.

As a frontend designer, not a developer, I'm intrigued by the techniques presented by the author, though most devs commenting here seem to be objecting to the code quality. (Way above my pay grade, but hopefully a solvable problem.)

As someone who loves to nerd out on creative processes, it's interesting indeed to contemplate whether AI assisted dev would favor waterfall vs incremental project structure.

If indeed what works is waterfall dev similar to the method described in TFA, we'll want to figure out how to use iterative process elsewhere, for the sake of the many benefits when it comes to usability and utility.

To me that suggests the main area of iteration would be A) on the human factors side: UX and UI design, and B) in the initial phases of the project.

If we're using an AI-assisted "neo waterfall" approach to implementation, we'll want to be highly confident in the specifications we're basing it all on. On regular waterfall projects it's critical to reduce the need for post-launch changes due to their impact on project cost and timeline.[1] So for now it's best to assume we need to do the same for an AI-assisted implementation.

To have confidence in our specs document we'll need a fully fledged design. A "fully humane", user approved, feature complete UX and UI. It will need to be aligned with users' mental models, goals, and preferences as much as possible. It will need to work within whatever the technical constraints are and meet the business goals of the project.

Now all that is what designers should be doing anyway, but to me the stakes seem higher on a waterfall style build, even if it's AI-assisted.

So to shoulder that greater responsibility, I think design teams are going to need a slightly different playbook and a more rigorous process than what's typical nowadays. The makeup of the design team may need to change as well.

Just thinking about it now, here's a first take on what that process might be. It's an adaptation of the design tecniques I currently use on non-waterfall projects.

----------

::Hypothesis for a UX and UI Design Method for AI-assisted, "Neo-Waterfall" Projects::

Main premise: Designers will need to lead a structured, iterative, comprehensive rapid prototyping phase at the beginning of a project.

| Overview: |

• In my experience, the DESIGN->BUILD->USE/LEARN model is an excellent guide for wrangling the iterative cycles of a rapid prototyping phase. With each "DBU/L" cycle we define problems to be solved, create solutions, then test them with users, etc.

• We document every segment of the DBU/L cycle, including inputs and outputs, for future reference.

• The USE/LEARN phase of the DBU/L cycle gives us feedback and insight that informs what we explore in the next iteration.

• Through multiple such iterations we gain confidence in the tradeoffs and assumptions baked into our prototypes.

• We incrementally evolve the scope of the prototypes and further organize the UX object model with every iteration. (Object Oriented UX, aka OOUX, is the key to finding our way to both beautiful data models and user experiences).

• Eventually our prototyping yields an iteration that fulfills user needs, business goals, and heeds technical constraints. That's when we can "freeze" the UX and UI models, firm up the data model and start writing the specifications for the neo-waterfall implementation.

• An additional point of technique: Extrapolating from the techniques described in TFA, it seems designers will need to do their prototyping in a medium that can later function as a keyframe constraint for the AI. (We don't want our AI agent changing the UI in the implementation phase of the waterfall project, so UI files are a necessary reference to bound its actions.)

• Therefore, we'll need to determine which mediums of UI design the AI agents can perceive and work with. Will we need a full frontend design structured in directories containing shippable markup and CSS? Or can the AI agent work with Figma files? Or is the solution somewhere in between, say with a combination of drawings, design tokens, and a generic component library?

• Finally, we'll need a method for testing the implemented UX and UI against the USE criteria we arrived at during prototyping. We should be able to synthesize these criteria from the prototyping documentation, data modeling and specification documents. We need a reasonable set of tests for both human and technical factors.

• Post launch, we should continue gathering feedback. No matter how good our original 1.0 is, software learns, wants to evolve. (Metaphorically, that is. But maybe some day soon--actually?) Designing and making changes to brownfield software originally built with AI-assistance might be a topic worthy of consideration on its own.

----------

So as a designer, that's how I would approach the general problem. Preliminary thoughts anyway. These techniques aren't novel; I use variations of them in my consulting work. But so far I've only built alongside devs made from meat :-)

I'll probably expand/refine this topic in a blog post. If anyone is interested in reading and discussing more, I can send you the link.

Email me at: scott [AT] designerwho [DOT] codes

----------

[1] For those who are new to waterfall project structure, know that unmaking and remaking the "final sausage" can be extremely complex and costly. It's easy to find huge projects that have failed completely due to the insurmountable complexity. One question for the future will be whether AI agents can be useful in such cases (no sausage pun intended).

It’s common practice to put your preferences and tips/advice into a readme solely for the LLM to consume to learn about what you want it to do.

So you’d set things like code standards (and hopefully enforce them via feedback tools), guides for writing certain architectures, etc. Then when you have the LLM start working it will first read that readme to “learn” how you want it to generally behave.

I’ve found that I typically edit this file as time goes on as a way to add semi-permanent feedback into the system. Even if your context window gets too large when you restart the LLM will start at that readme to prime itself.

That’s the closest analogy I can think of.

I agree 100%, but in this very specific case, I really just wanted a working one-off solution that I'm not going to spend much time on going forward, AND I wanted to use it as an excuse to see how far I can go with AI tooling in a tech stack I don't know.

That being said, using AI as a teacher can be a wonderful experience. For us seniors, but also and probably more importantly, for eager and non-lazy juniors.

I have one such junior on my team who currently speed-runs through the craft because he uses AI to explain EVERYTHING to him: What is this pattern? Why should I use it? What are the downsides? And so on.

Of course I also still tutor him, as this is a main part of my job, but the availability of an AI that knows so much and always has time for him and never gets tired etc is just fantastic.

With this project, I was really only interested in the resulting application, and intentionally not in the resulting code.

I really wanted to see how far I can get with that approach.

I will ask it to clean up the code and its comments and report back.

with the ai age, i think its also making sure team members use ai in similar was to get repeatable results

True, this needs to be fixed.

What I wanted to express was that I didn’t do any of the implementation, that is, any logic.

I need to phrase this better.

I've taken this feedback to heart and tried to improve the structure a bit.

The main message I want to bring across is two-fold:

1. Senior developers are in a great position to make productive use of AI coding tools

2. I have (so far) identified three measures that make AI coding sessions much more successful

I hope the reworked version makes these more central and clear. I'm not a native English speaker, thus it's probably not possible for me to end up with an optimal version.

Still, I hope the new approach works a bit better for you — would love to receive another round of feedback.

We're just in a transitional moment. It's not realistic to expect LLM capabilities to leapfrog from marginally better autocomplete to self-guided autocoder without passing through a phase where it shows tantalizing hints of being able to go solo yet lacks the ability to follow through. Over the next couple years, the reliability, versatility, and robustness of LLMs as coders will steadily increase.

I dunno, I just had Copilot sneak in a typo today that took about ten minutes of debugging to find. I certainly could have made a similar typo myself if copilot hadn't done it for me, but, all the same copilot probably saved me a minute worth of typing today but cost me 10 minutes of debugging.

Is it really 5x? I'm more surprised about 25+ years of experience, and being hard pressed to learn enough python to code the project. It's not like he's learning programming again, or being recently exposed to OOP. Especially when you can find working code samples about the subproblems in the project.

This is actually a pretty compelling reason for me to suggest that my company not hire consultants/contractors to write code for us. A ton of our dev budget is already spent on untangling edge-case bugs from poorly written/understood code.

This is a great observation and a beautiful perspective.

Would it be okay for you if I quote this on a revised version of the article (with proper attribution, of course)?

I think with AI my productivity had increased 1% at most. If I measure time saved per week.

Same 3 is the only use case I've found that works well enough. But I'll still usually take a look on google / reddit / stackoverflow / books first just because the information is more reliable.

But it's usually an iterative process, I find pattern A and B on google, I'll ask the LLM and it gives A, B and C. I'll google a bit more about C. Find out C isn't real. Go back and try other people commenting on it on reddit, go back to the LLM to sniff out BS, so on and so on.

This is interesting. Do you run a custom app in house? What are some of the main functions of the app? Internal or customer facing?

But can it center a div?

At all the jobs I had, the valuable stuff was shipped features. The baseline was for them to work well and to be released on time. The struggle was never writing the code, it was to clarify specifications. By comparison, learning libraries and languages was fun.

I don't really need AI to write code for me, because that's the easy part. The aspect that it needs to be good at is helping me ship features that works. And to this date, there's never been a compelling showcase for that one.

Have you considered that the opposite explanation might be true instead?

It could be that other developers are not benefitting from AI as much as you because they don't understand it.

It could also be that you are benefitting more than them because you're less skilled than them, and AI can fill in your gaps but not theirs.

That's the problem. The new norm will be 10x of pre-AI productivity, nobody will be able justify hand-writing code. And until the quality bar of LLM's/their successors get much better (see e.g. comments above looking at the details in the examples given), you'll get accumulation of errors that are higher than what decent programmers get. With higher LOC and more uninspected complexity, you'll get significantly lower quality overall. The coming wave of AI-coded bugs will be fun for all. GOTO FAIL;

Giving your anecdotal experience is only useful if you include anecdotal context: seniority, years of experience, technology, project size, sprint goal complexity…

Can you talk through specifically what sprint goals you’ve completed in an afternoon? Hopefully multiple examples.

Grounding these conversations in an actual reality affords more context for people to evaluate your claims. Otherwise it’s just “trust me bro”.

And I say this as a Senior SWE who’s successfully worked with ChatGPT to code up some prototype stuff, but haven’t been able to dedicate 100+ hours to work through all the minutia of learning how to drive daily with it.

> The gap between Sr and Jr skills with close as AI gets better.

Sorry, but as the wind blows, we only have indications of the opposite so far and it's wildly reported as a generating losing their ability for deep thought, relying too much on automated code generation. This is especially damaging for juniors, who never get the chance to learn the tricks of the trade in the first place, and they're the ones having a hard time explaining what the code the LLM generated even does.

Seniors will be few and far between, and we will charge premium for fixing up the crap LLM created, which will fit us great before retirement.

Good luck when we are retired.