JetBrains has released Rider 2024.1. This release contains a new monitoring tool window and a collection visualizer. There also appeared the plugins: .Net Aspire, AI Assistant and features regarding debugging.
By Robert KrzaczyńskiThis week OpenAI gave us access to GPT-4o (“o” for “omni”), which aims to compete better in speech recognition and conversation. It’s is almost certainly a stronger LLM, too. But can it do a code review? I put it through some tests to find out.
To start with, I checked if it could do long multiplication:
This is correct, as you can confirm on any online calculator. Note that it doesn’t attempt to “work through” the result for you — it has just passed it to a calculator. The “view analysis” icon at the end basically confirms this. Finally! This has failed to work on previous models.
I have no personal interest in formal code reviews. Yes, sometimes a junior developer might actually demand one, but unit tests and pairing sessions will usually suffice in sanding away bad code practice. But a few mumbled comments can help indicate mild unease with code, without appearing to be imposing some kind of authoritarian order over easily rewriteable snippets. In short, a code review is more of a social interaction than anything. Focus on engineering issues, not code semantic detailing.
However, code reviews are a good LLM use case for developers. Here I’ll look at three different bits of code, each of which simply draws a box with a diagonal line. I’ll add problems and see how GPT-4o (via ChatGPT) reviews them.
Here we start with our simple diagonal line routine in C#:
using System; public class Program { public static void Main() { BoxWithDiagonalLine(5); BoxWithDiagonalLine(6,1); BoxWithDiagonalLine(6,-1); BoxWithDiagonalLine(2); BoxWithDiagonalLine(-1); } // Example 1 public static void BoxWithDiagonalLine(int boxsize, int linepos = 0) { for(int i=0; i < boxsize; i++) { for (int j = 0; j < boxsize; j++) { if (j == linepos) Console.Write("x "); else Console.Write("o "); } linepos++; Console.Write("n"); } Console.Write("n"); } }
You can throw this into https://dotnetfiddle.net/ (set the compiler to .NET 6 at least, though) and see the responses:
This is fairly simple, and relatively self-explanatory. An outer loop controls the rows, an inner loop places the “x” in the right place. The variable naming is minimal, but the code is short. There is a default value for the method to give a full diagonal line from top left to bottom right. Whereas we can move the start line position left or right, it doesn’t make much sense to have a negative boxsize. So we could improve it by turning the method parameter to an unsigned int
. This would immediately stop the last example from even compiling. A wildly large linepos will not be visible, but it isn’t dangerous.
So far, there is little to comment on.
In the second example, I will replace the for
loops with while
loops:
// Example 2 public static void BoxWithDiagonalLine(int boxsize, int linepos = 0) { int i = 0; while(i++ < boxsize) { int j = 0; while (j++ < boxsize) { if (j == linepos + 1) Console.Write("x "); else Console.Write("o "); } linepos++; Console.Write("n"); } Console.Write("n"); }
Now this works, but deserves a slight eyebrow raise. By post-incrementing the loop variables (the ++
after the i and j mean the values are read first before they are increased by one) we save some space, but we need that ungainly linepos + 1
expression. A while
doesn’t express the beginning and end state quite so nicely as a for
loop does — but we are not sliding into any madness.
We make one last change. We now have a variation that does the wrong thing. It tries to use linepos to control the outer loop:
//Example 3 public static void BoxWithDiagonalLine(int boxsize, int linepos = 0) { while(linepos < boxsize) { int j = 0; while (j++ < boxsize) { if (j == linepos + 1) Console.Write("x "); else Console.Write("o "); } linepos++; Console.Write("n"); } Console.Write("n"); }
In an unnecessary attempt to remove one loop variable, the box itself will no longer be complete — although it will work with the default value. In fact, this solves a different problem, always reliably finishing with a cross in the bottom corner.
This poor code could probably come about because the developer continued to use a while
inappropriately.
So let’s summarize the way we can look at the different versions:
while
loop, which doesn’t quite have the certainty of a for
loop. It also makes for an ugly expression to take into account the post-increment.What do we expect from an AI code review? First of all, I have given it no comments, so will it work out what the code does? Only the name of the method reveals this.
Here is GPT-4o response:
“Your code is functional and effectively generates a box with diagonal lines based on the provided parameters. However, there are a few improvements and edge cases to consider:
linepos
: Improve handling of diagonal positions to ensure the diagonals are correctly drawn when linepos
is negative or greater than the box size.So it kind of did understand what the code did — even if it thought the program drew “lines” not a “line”. This is still very good.
The code it suggested was the following:
// ChatGPT4o suggested code public static void BoxWithDiagonalLine(int boxSize, int linePos = 0) { if (boxSize < 2) { Console.WriteLine("Box size must be greater than 1."); return; } // Adjust linePos if it's out of the box boundaries if (linePos < 0) linePos = 0; if (linePos >= boxSize) linePos = boxSize - 1; for (int i = 0; i < boxSize; i++) { for (int j = 0; j < boxSize; j++) { if (j == linePos) Console.Write("x "); else Console.Write("o "); } linePos++; // Reset linePos if it goes out of the box boundaries if (linePos >= boxSize) linePos = 0; Console.Write("n"); } Console.Write("n"); }
Here are the results of putting this into dotnetfiddle.net:
I think most people would be happy with the size less than two guards. It also camel-cased the params nicely, as this is right for C#. Unfortunately, it missed the idea of changing the position to allow for the line to start lower down. And by resetting the linepos, we get that unfortunate error at the bottom of the second example.
However, as code review, it hit the right note.
Let’s feed it the second example and see how it does. Let me pick out the interesting comments:
“The use of while
loops is functional but could be less intuitive compared to for
loops for such iterations.”
Excellent. This is exactly the right note.
“The condition j == linepos + 1
is used to determine where to place “x”. This works but can be made more intuitive by aligning with the index i
.”
Again, this is good. It understands the difference between functionality and readability.
The produced code clamped the value of linepos, and rather oddly used the modulo function to change the value. But it ended up with the same result. It reverted to for
loops, which is excellent.
Finally, let’s deliver the bad Example 3, and see how it fairs. Oddly, it didn’t spot the bug, but did suggest the following:
“Edge Cases:
– When linepos
is negative, it works but may not be intended.”
This is an indication that it didn’t quite understand the code — although, again, I never explicitly stated what I was doing. So perhaps a wrapping line may have been a reasonable guess?
Overall, I was quite impressed with the nature and the delivery of the review. The code GPT-4o produced was less promising, but it did ok. There was no “great clomping foot of nerdism” in the criticism of the code; instead, it gently nudged the reader in a better direction.
This is perhaps not where I expected to end up — with a good attitude but slightly inaccurate code. However, I would be happy to let this review be seen by a nervous junior; in that respect it does a slightly better job than some humans.
The post Reviewing Code With GPT-4o, OpenAI’s New ‘Omni’ LLM appeared first on The New Stack.
Microsoft recently released the new Phi-3-mini SLM (small language model), and I previously wrote about running Phi3 locally using Ollama. I’ve been exploring the world of hosting AI models in code lately, and this stuff is getting really interested. It turns out that among the tools to run AI models locally, there are developer tools […]
The article Build a Generative AI App in C# with Phi-3-mini LLM and ONNX appeared first on Build5Nines.