Follow-up regarding existing todos

[Adakole2020]

Hello @jmdevall , was trying to look for a more effective way of reaching you but this was the only way available and should suffice. I am curious on how much progress you were able to make regarding fixing the identified issues in the todo list. I would be more than happy to collaborate on any pending tasks. However, I understand you may have already investigated this issue so would have more context on what may need to be done to resolve those todos much quicker.

Any response you can provide would be key towards integrating the work you have already done in the best way possible.

[jmdevall]

Hello! I’m glad you’re interested in integrating this work into your project and reusing it. Please excuse my late reply, but I don’t usually log into GitHub.

First, I’d like to remind you that this project is experimental, without any pretensions, and I’ve completely abandoned it for now. At the beginning, I approached it with enthusiasm because the paper seemed very interesting to me, and I wanted to experiment on my own.

Keep in mind that this is from several years ago. At that time, the most advanced model available was ChatGPT 3. This was before the rise of agents.

Looking back with perspective, I believe that much of what was presented in the paper could potentially be improved using agents and tools instead of pure prompting, although the essence of the idea remains the same.

Obviously, a command-line interface isn’t very user-friendly, but it was the quickest way to create a minimal product. Today, the use case I envision is for this to function as an advanced "tool" to be used by agents.

• Back then, there were no utility libraries available to connect with large language models (LLMs). If you look at the implementation of the connector for the LLM (the package adapter.out.llm), I was using the raw REST API and specifically the "completions" API. By doing it this way, you need to format the prompt according to the specific model to separate the messages. Today, it would be much more appropriate to use a conversational API with a library that already implements all of this (like LangChain4j). This would be straightforward to implement.

The most challenging and valuable part of the project is the Java parser classes, which are also the most complex. In this regard, there are some tests that can help you understand how each part works separately. In the testbench package, you can see dummy classes that use the tests to try parsing and extracting information. I suggest also looking elsewhere: Eclipse IDE has implemented refactoring tools that should use parsers to create a dependency tree between code parts, similar to what I’m creating. The code from Eclipse is likely more robust than mine. If your goal is to create a serious product rather than just experiment, I’d recommend looking for and reusing something that’s been well-tested.

About the TODOs List:

• "bug in constructed-by relations in fields (fields are pruned. Search alternative..."
  I don't recall what I was referring to when I wrote this, and after reviewing the code, I haven't found anything that helps me remember... perhaps it's related to the TODO in the DependencyGraph class.

Node cu=forest.get(nodeid.getFile()); //TODO: bug. el nodo de tipo field se ha recortado y no encuentra

• "The algorithm is executed on another java project. Maybe I should create a folder within the project"....
  Well, I was actually referring to setting up some kind of directory infrastructure where there is a set of test projects, so I could test the algorithm in a more controlled environment, rather than pointing to an external repository.

• Implement the oracle. The oracle is an external component that, once the algorithm has made changes to the code, tells it what might be wrong.
  Two possible implementations come to mind:
  A human who manually reviews the code (Human in the loop).
  Running the project's tests after the changes and providing the results. For this second option, if it's a Maven project, you would invoke "mvn test" from Java on the project and obtain the test results.

• "escaping object analysis": Before reading the paper, I didn't know what this was. It refers to analyzing whether an object returned by a method can affect the code that invokes it. Performing this analysis to determine if it can or cannot affect the code helps decide whether to continue with the algorithm or not. Currently, it always performs the analysis just in case. In an initial version, this isn't critical because if it doesn't affect the code, nothing goes wrong, and at most, it's slightly less efficient. Back then, I remember researching how to implement this, and it was indeed very complicated..

• The code responsible for mixing...
  These are complications related to mixing the code generated by the LLM with our own code. As you can see, I've added a dependency with java-diff-utils. This is a library used to compare two texts and extract the chunks that are differences between the two texts. The paper proposes merging what the LLM returns with the code, using something like this to remove parts of the class sent to the LLM.

The truth is, I tried applying this approach, but it's very complex to implement. Today, given that the context size of LLMs has grown and is no longer such a big issue, I believe it's not worth complicating things. It's better to give the entire class to the LLM and tell it to apply the changes by printing the class again.

One of the changes compared to what the paper indicates is that I directly recreate the complete tree after the change, instead of trying to update the tree with the change. Although recreating the complete tree is more inefficient, it is much simpler for me. I don't know if this has any downside that hasn't been considered.

// DependencyGraph dp = d.updateDependencyGraph(labels, fragment, newFragment, bi.getB());
 DependencyGraph dp = dependencyGraphConstructor.constructDependencyGraph(r); //isn't it more easy???