exclusion should trump vectorise?

[deas]

I think .vectorcode exclusion should trump vectorise.

Assume we have a repo using inclusion/exclusion configuration. Calling vectorcode vectorise respects the exclusions and indexes the whole repo. However, calling vectorise on an excluded file does not respect this and this file ends up in the index. Specifically, I found this surprising, as it was kicking in via git hooks. ;)

I think we should write a message, letting the using know that exclusion trumped the vectorise call.

[Davidyz]

calling vectorise on an excluded file does not respect this and this file ends up in the index

This is not supposed to happen, regardless of whether the vectorise call is followed by file paths or not. When calling from the command line, the blacklisting (exclusion) happens after the internal list of files is populated, so it should always exclude files regardless of whether you supplied the file paths or not. Would you be able to run the "faulty" vectorisation with logging enabled, and drop the log here?

[deas]

Meanwhile, I guess I remember where I picked this up 🤣.

[deas]

Got lost with regards to inclusion and exclusion, tbh. Just .gitignore can already get confusing once order and ! exclusions come into play.

How would I express "Only ever vectorise src/**/*.clj and test/**/*.clj"?

[Davidyz]

How would I express "Only ever vectorise src/**/*.clj and test/**/*.clj"?

I believe the following will do the trick:

# vectorcode.exclude
*
!src/**/*.clj
!test/**/*.clj

[deas]

It does indeed. Thanks!

Why did (or not) you chose to not go with a .vectorcodeignore, leverage pure git ignore semantics and a library?

Just asking out of curiosity. Not asking you to change this.

[Davidyz]

Why did (or not) you chose to not go with a .vectorcodeignore, leverage pure git ignore semantics and a library?

It's personal and opinionated, really. I, too, find gitignore confusing when it contains more than wildcards, and I find it easier to do it this way (in the sense that I have the option not to negate things again and again). Also, the current implementation does use a gitignore parser.

On the other hand, by splitting the include and exclude specs and making the exclude spec take precedence, it actually offers more flexibility. Imagine using this on a large repo (hundreds or even thousands of files), and the maintainers added all files under src into include the spec, but chose not to include an exclude spec. This would allow the collaborators to keep their own exclude spec that only vectorise a subset of code in src (for the submodule they're working on, for example), which makes a lot of sense if some of them have a weaker PC that struggles to query the whole repo.

To be completely honest, a lot of the decisions that I made in this project were opinionated. I didn't expect this project to draw so much attention, and I was like "oh this is prob not the best way to do it, but for a toy/personal project it's fine".