CPPBridge is a library that allows Pharo to share memory with any C/C++ application. Opening the door not only to IPC and data sharing but also even complete control of C++ code from Pharo and vice versa.
CPPBridge class has a comment with all the details, each method at class side is commented , there is also an example method . C++ files required for the example files are included with the repo, you have just to compile them and execute Atlas-server.
In a few hours it should be available from Package Browser, if you like to install it manually then open a Playground inside Pharo and do the following command
Metacellonewbaseline:'CPPBridge' ; repository:'github://kilon/CPPBridger:master';load.You need to first build atlas-server.cpp which you can do with
g++ atlas-server.cpp -o atlas-server
then execute it
./atlas-server
it will ask you to type a string, type whatever you like and press enter. Will provide information about the C struct which is used to store data into the shared memory. Basically this application is opening / creating a file called "mmaped.bin" which is just a simple binary file and maps it to the memory. That means that whatever is in the file is loaded to the memory.
Then shares the memory with any other process who will memory map the same file.
Then we start Pharo , install CPPBridge to it and go to CPPBridge class >> retrieveSharedValueStep1 , read the comment and click the green triangle to execute so that you receive the C string you typed from Pharo and then execute retrieveSharedValueStep2 to unmap and close the file
Bellow is the source of both methods together with their comments.
retrieveSharedValueStep1
<example>"This method is an example that retrieves a struct from a shared memorysection, in order for this example to work you need you first copypaste the contents of the Example Souce Code of the C++ file in thecomment section (you can also find the cpp file in the same directorywhere the git repo has been downloaded) of this class to a C++ sourcecode file and compile it a run then replace the path of in this codeof CPPBridge openFile: with the correct path of the bin that the C++files has created , in order for this to work also you need to executethe C++ example first so it creates and file and share the memory.After executing this method you can execute retrieveSharedValueStep2to unmap and close the memory mapped file (keeps sharing the memory itjust does not store it to the file)"
|instancefdNumberlseekmmapPointerdatastruct|
"Let's create an instance just an an example but we wont use itbecause we can use either class method or intance methods. You wouldwant to use instance method if you want to open multiple memory mappedfiles meaning multiple areas of shared memory. Class methods for usingjust one"
instance :=CPPBridgenew.
"Warning !!! You must change the path to the file that is located inyour hard drive. The file should be at the same location you builtatlas-server.cpp which is responsible for creating the file. Thenumber returned is a number that OS uses to identify the image , flagO_RDWR is just a number that states that we want to write and read thefile"
fdNumber :=CPPBridgeopenFile:'/Users/kilon/git/Pharo/Atlas/mmapped.bin'flags: (O_RDWR) .
"lseek is used to stretch the file to a new size" lseek :=CPPBridgelSeek_fd: fdNumber range:3999value:0.
"this is the most importan method, this method maps the file tomemmory , which means it loads its contents into memory and associatesthe memory with the file. PROT_READ means we want to write the memory, PROT_WRITE to write the memory and MAP_SHARED is the most importanbecause it defines the memory area as shared so we can access it fromother application"
mmapPointer :=CPPBridgemmap_adress:0fileSize:4000flag1: (PROT_READ|PROT_WRITE )flag2:MAP_SHAREDfd: fdNumber offset:0 .
"This assigns the pointer to our Pharo structure so we can use it toget the contents of the C structure located in the shared memory"
struct :=CPPStructpointTo: (mmapPointer getHandle ).
"data here serves as a convenience array its not necessary we use itjust to collect information about the instance, the fd number of thefile, the streched size of the file, the adress (point) where the fileis mapped to in memory and structthat contains the values of the Cstruct that we received" data :={ instance. fdNumber. lseek. mmapPointer . struct}. data inspect.
"Store data to the class so we can use it in the second method"ExampleDATA:= data. ^data
" Its also possible to write to the shared memory , in this case weuse once again the C struct which has the following members(variables) :1) data = char[3000] this is where we store the string 2) count = int this is where we store the size of the stringstruct := {(mmapPointer getHandle copyFrom: 1 to:3000 )asString . (mmapPointer getHandle integerAt: 3001 size:4 signed: false)}.mmapPointer is the pointer that points to the first byte of the sharedmemory.getHandle gives us the memory address that the pointer points tocopyFrom:1 to:3000 copies byte from byte 0 (remember C counts from 0 ,Pharo counts from 1) to byte 3000 because the string we store isstored as a char array of 3000 elements, each element is a char, eachchar is 1 byte in legth and represents a single character of thestring. This gets the value of the first struct member.on the other hand integerAt: 3001 size: 4 signed: false returns us thevalue count member of the C struct . its an integer in position 3001because our string is a char[3000] and the size is 4 bytes because itsan C int, signed false because we use no negative values because itdoes not make sense for a string to have negative length. This getsthe value of the second struct member" retrieveSharedValueStep2 <example>"this one is to be triggered after retrieveSharedValueStep1 when we arefinished with the shared memory and we want to erase it and close the memory mapped fileExampleDATA here severs as convenience array for storing the returnvalues from the call to C function.fd -> a number with which the OS identifies the memory mapped filelseek -> a number that returns the size of the file after we stretchedit to its finaly sizemmap -> this is the juice, the pointer that points to the first byteof the shared memory. The final byte is of the shared memory definedby the size of the file.struct -> here we put the values we retrieved from the C struct whichwas the goal of the previous methodExampleDATA = { CPPBridgeinstance . fd . lseek . mmap . struct}""first lets unmap the file, basically this will disassociate the filewith the memory area that is assigned to ,meaning the OS will no longerstore in that file automatically what we change in shared memory"CPPBridgemunmap_data: (ExampleDATAat:4) filesize:4000.
"And because we wont be using the file anymore lets close. Rememberyou still have access to the shared memmory. "CPPBridgecloseFile: (ExampleDATAat:2). ```
-----
# Why bother making such a library ? In my saga to find a way to use Pharo as a scripting language for UnrealGameEngine, I had two options
1. BuildUnreal as a Libraryand use PharoUFFI to launch and control it
1. EmbedPharo inside the UnrealExecutable (this is what every other scripting language uses for controlling Unreal) Option a was a no go, because Unreal is huge , complex and uses its own custom made build tools, making a DLL for Pharoor an army of DLLs out of the question
Option b EmbedingPharo inside an executable is impossible and implementing it also insanely complex.Naturally my mind went first into sockets which is what I use to make Pharo able to use Python libraries. However sockets have proven too slow for the insanely fast loops of Unreal. -----
# What are the advantages ?
*No need to move data around. Sharing memory means you don't have to move data around, you can use directly the shared memory* Extend the Pharo image beyond Pharo. Shared memory is mapped into a file means that you can do with C++ what you can do with Pharo image, save the live state directly to a binary file. That means we no longer need to worry about sessions and reinitializing C/C++ data since memory mapped file acts as an extension of the Pharo image. * Blazing fast. Memory mapping is a function that comes directly from the OS kernel and as such it has to be extremely fast. Memory mapping is also what is used for dynamically linked shared libraries an extremely important feature for any application including Pharo that heavily depends on (see Cairo for Athens). So its a very mature , well tested method. * No extra libraries needed to be installed, CPPBridge uses OS libraries to work out of the box* Low level handling of memory. Direct access to memory you can even manipulate the memory byte by byte* Memory efficient. Memory mapping excels at large data, the bigger the better. Can take full advantage of your entire free memory and not waste a byte. That means also that can be used to optimise Pharo memory, since you could compress your Pharo objects to bytes and mapped file will store the live state.* Tons of Languages. Because memory mapping is a core functionality for every OS out there, pretty much every programming language supports it. CPPBridge currently supports only C/C++ but all languages can be supported giving access to Pharo to any library for any programming language. Sky is the limit* Self Documented. CPPBridge is small, simple and with large class comment and comments for every method. YouTube video tutorial also available and linked on top. * Works both ways, C/C++ and Pharo can access and modify the shared memory. Making it possible for C/C++ to use Pharo libraries and Pharo to use C/C++ libraries.* Experiments have proven that it improves sex life... if it does not please file a bug report ;)-----# What are the disadvantages ?* Candy Crash Saga. Dare do something incorrectly and Pharo will crash. CPPBridge can easily point to wrong address if you are not aware of what you doing. * C++/C . If you think you can avoid learning C/C++ and that this is a magic solution , think again. The C/C++ application must be modified to include shared memory mapping for CPPBridge to work .* Local only. Unlike sockets, Shared Memory works only on the same machine so no remote execution and manipulation of code like in my socket bridge to Python* UFFI still No1 option. No replacement for UFFI it actually depends on it to work , so if you can turn your C/C++ code into a DLL that should be your first option.-----# Roadmap Currently CPPBridge works only on MacOS , most likely on Linux too (because it uses the Unix architecture) but I will have to test it.Windows is coming next ASAP, since its my No1 platform for creating commercial games.Maybe establish a small protocol of communication via the Shared Memory , JSON looks like a good universal format -----# ThanksBig thanks to Eliot for inspiring me and Esteban for helping me figure out things.