ClinchedDBValues class

[yakra]

• C++: Clinched values are constructed, then copied and moved:
  • Copied when passed to the add function by value
  • Moved when we push_back onto the value containers
• Instead, pass the add functions the args by which to construct, and emplace_back.
  Or not. We'd still construct the string via the + operator, and move it to the container.
  • Why not just pass the strings to add_cfoo as an rvalue reference.
• Consider using pointers, to cut down on copy operations when resizing the vector.
  Or not. It's move instructions that are involved here, which are already very low cost (as I see from my attempts to store args rather than strings). Pointers would just be more trouble than they're worth.
• Longer term, look into storing the args themselves, not full strings, and saving RAM.
  • How much RAM does this whole structure take up anyway?
  • Could a similar arrangement also benefit Python slightly? Would need to rewrite as a class; it's just 3 lists currently. A pretty simple task though.

[yakra]

Wow, OK. Why am I getting slower results when passing as an rvalue ref instead of by value?

[yakra]

How much RAM does this whole structure take up anyway?

HighwayData @ 27a625459516a8f5c53f73a24ce51416ab991677
UserData @ 3ec63996d33e499027a3d4c8f47abc0d3ce7659f
I get a rough 49,663,030 via

size_t size(bool verbose)
{	if (verbose) std::cout << "ClinchedDBValues::size:\n";

	size_t csmbr_size = sizeof(csmbr_values);
	for (std::string& csmbr : csmbr_values) csmbr_size += csmbr.size()+1;
	if (verbose) std::cout << "csmbr:\t" << csmbr_size << " not counting node size\n";
	csmbr_size += csmbr_values.size() * sizeof(void*)*2;
	if (verbose) std::cout << "csmbr:\t" << csmbr_size << " assuming node = (pointer,object,pointer)\n";

	size_t ccr_size = sizeof(ccr_values) + sizeof(std::string)*ccr_values.size();
	for (std::string& ccr : ccr_values) ccr_size += ccr.size()+1;
	if (verbose) std::cout << "ccr:\t" << ccr_size << std::endl;

	size_t cr_size = sizeof(cr_values) + sizeof(std::string)*cr_values.size();
	for (std::string& cr : cr_values) cr_size += cr.size()+1;
	if (verbose) std::cout << "cr:\t" << cr_size << std::endl;

	size_t total = csmbr_size + ccr_size + cr_size;
	if (verbose) std::cout << "total:\t" << total << std::endl;
	return total;
}

[yakra]

Longer term, look into storing the args themselves, not full strings, and saving RAM.

Part of the idea here was to avoid copying strings when resizing the vectors, having forgotten this would only involve moving, not copying.

cr_values & ccr_values:
These were converted into a class, so as to only have to resize a single vector.

• We do see a small gain from not having to construct our strings. However, with the strings constructed single-threaded later on when writing the SQL file, that should be a net loss in performance.

csmbr_values:
Surprisingly, converting to a class & avoiding string construction harms performance.

• Could it be that in this instance, adding to a list is more expensive than adding to a vector? We have to update pointers on the new element, the old final element, and on csmbr_values.end() at the very least...
• Will it be any faster if I change csmbr_values to a vector?
  Looks like a nope; lab1& lab1.5 are still slower than leaving csmbr_values alone; lab2 & lab3 expected to be similar. BiggaTomato within margin of error.

.sql file:

• What's our performance hit from string construction so far? Test with threading disabled to isolate the relevant data.
• Try inserting the elements of a cr_val directly into the ofstream, along with the necessary chars
  • On the one hand, this avoids string construction.
  • OTOH, we'll have 7 insertions rather than 1. Could this be a net performance hit?

---

Best case, this is just chasing after diminishing returns...

[yakra]

Time savings would be a fraction of a second, if anything at all. No need to go down this rabbit hole. Let's keep the code simple.