Parallelize File Processing - #80
Merged
Merged
Conversation
Added support for multi-threading in the main file processing loop of rcnCount(). Added boolean useMultiThreading member flag to RcnStatOptions type to allow API users to set whether MT should be enabled or disabled. Added Slice type to bind start and end index for an array into one data structure. Added --no-parallelization CLI option to scount to disable MT on demand. By default, MT is enabled for scount. The threading-related functions are put in the new internal threading.h header. The accompanying threading.c implement all cross-platform threading-related capabilities. For platform-dependent threading code, the platform-specific threading.c files are added, containing native code for GNU/Linux with pthreads and Win32 with the standard Windows API using CreateThread(). [CL]: Added parallelization capabilities. [Issue#9] Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Fixed compilation errors. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
…c_bool. Using atomic type instead of manually locking and unlocking on an initialized mutex for improved performance. The mutex and its init deinit are kept around, even though not used anymore. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Non-locking atomics are supported by the MS implementation. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Fixed wrong member variable name. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
…coverage enabled. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
…ctions. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Changed to use LPCRITICAL_SECTION instead of CRITICAL_SECTION pointer. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
…ild. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Added cast to LPTHREAD_START_ROUTINE for lpStartAddress argument in call to CreateThread(). Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
…ntrol.mutex to NULL. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Reverted commit cb3dd46 Reason: Readability was actually worse, might be confusing to deinit an object where init failed, even though it is safe in this context. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Changed to use if-statement instead of weird switch-case. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Changed to provide richer error information. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Added debug assertions for WaitForSingleObject() and CloseHandle() return status values. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
…nction. Doesn't test much, so removed. Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
Signed-off-by: Phil Gaiser <phil.gaiser@raven-computing.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Adds parallelization capabilities.
Adds support for multi-threading in the main file processing loop of
rcnCount().Adds boolean
useMultiThreadingmember flag toRcnStatOptionstype to allow API users to set whether multi-threading should be enabled or disabled.Adds
--no-parallelizationCLI option to scount to disable multi-threading on demand.By default, scount uses multi-threading as basically all workstations these days have multiple cores. This will improve performance for the end user by default.
Threading on GNU/Linux uses
pthreadswhile on Windows we use the standardCreateThread()from the Windows API. Implementations that are not cross-platform are placed into the corresponding platform-specific directories.Adds
Slicetype to bind start and end index for an array into one data structure.