Building RetroShare on Mac OS
A couple of weeks ago RetroShare has released version 0.6.6. But while download page offers quite a variety of pre-built packages for different operating systems, there is no 0.6.6 build for Mac OS (only 0.6.5 is available).
I found this peculiar, so I decided to try to build it from sources myself. It also seemed like a good opportunity to finally try RetroShare, as it has been available for more than 15 years, and I’ve heard many good things about it.
At the moment Mac OS section on download page looks like this:
Let’s take a cheeky look into the building process then.
RetroShare is based on Qt and currently it uses
qmake. So as the very minimum, aside from cloning the repository, you need to have Qt installed in your system. If anything, I built mine from sources, but it will most likely just as well work if you’ll get it using official installer.
Just in case, here’s my environment:
$ sw_vers -productVersion 10.15.7 $ xcrun --sdk macosx --show-sdk-version 11.1 $ gcc --version Configured with: --prefix=/Applications/Xcode.app/Contents/Developer/usr --with-gxx-include-dir=/Library/Developer/CommandLineTools/SDKs/MacOSX.sdk/usr/include/c++/4.2.1 Apple clang version 12.0.0 (clang-1184.108.40.206) Target: x86_64-apple-darwin19.6.0 Thread model: posix InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin $ qmake --version QMake version 3.1 Using Qt version 5.15.2 $ cd /path/to/retroshare/src $ git show 751fffc30 (HEAD, tag: v0.6.6)
The base application is extendable with plugins, and there is a
qmake configuration option for building those too. I’ll describe the process for both variants: with and without plugins.
Building without plugins is easier, because it has less 3rd-party dependencies. And I would say, 3rd-party dependencies is the main difficulty of building RetroShare from sources.
There is some documentation on the matter (more about that later), from which I got to know that I need to install the following dependencies:
$ brew install openssl $ brew install miniupnpc $ brew install libmicrohttpd
SDK resolving issue
Let’s try to configure it now:
$ cd /path/to/rapidshare/src $ mkdir build && cd $_ $ qmake .. Project MESSAGE: RetroShare version 0.6.6-0-g751fffc30 determined via git Project WARNING: QMAKE: You have disabled deprecated warnings. Project WARNING: QMAKE: You have disabled C preprocessor warnings. Project WARNING: QMAKE: You have enabled RetroShare direct chat which is deprecated! Project MESSAGE: ***retroshare.pri: Set Target and SDK to MacOS 10.11 Project MESSAGE: ***retroshare.pri:MacOSX Project ERROR: Could not resolve SDK SDKVersion for 'macosx10.11' using --show-sdk-version
So right from the very beginning you are getting this fantastic error in your face. Smells like something is hardcoded somewhere.
And indeed, if you open
/path/to/retroshare/src/retroshare.pri, there is this weird block at the line 129:
# To select your MacOsX version append the following assignation to qmake # command line "CONFIG+=rs_macos10.11" where 10.11 depends your version macx:CONFIG *= rs_macos10.11 rs_macos10.8:CONFIG -= rs_macos10.11 rs_macos10.9:CONFIG -= rs_macos10.11 rs_macos10.10:CONFIG -= rs_macos10.11 rs_macos10.12:CONFIG -= rs_macos10.11 rs_macos10.13:CONFIG -= rs_macos10.11 rs_macos10.14:CONFIG -= rs_macos10.11 rs_macos10.15:CONFIG -= rs_macos10.11
This hardcoded list of versions doesn’t look good at all. And no matter what values I provided to
CONFIG, the configuration was always failing with the same error. Certainly can’t tell what grand plan original developer had behind it, but it definitely does not make sense to me. So I ended up editing this block manually like this:
macx:CONFIG *= rs_macos11.1 #rs_macos10.8:CONFIG -= rs_macos10.11 #rs_macos10.9:CONFIG -= rs_macos10.11 #rs_macos10.10:CONFIG -= rs_macos10.11 #rs_macos10.12:CONFIG -= rs_macos10.11 #rs_macos10.13:CONFIG -= rs_macos10.11 #rs_macos10.14:CONFIG -= rs_macos10.11 #rs_macos10.15:CONFIG -= rs_macos10.11
11.1? Because that’s what
xcrun reports in my system, and apparently that’s what is getting checked against during configuration.
Now the project configures fine, so we can build:
$ qmake .. $ make -j12
Missing OpenSSL headers paths
But during the build I got these errors:
../../../openpgpsdk/src/openpgpsdk/packet.h:32:10: fatal error: 'openssl/bn.h' file not found #include <openssl/bn.h> ../../../libretroshare/src/pqi/pqihash.h:26:10: fatal error: 'openssl/sha.h' file not found #include <openssl/sha.h> ../../../retroshare-gui/src/TorControl/CryptoKey.cpp:41:10: fatal error: 'openssl/bn.h' file not found #include <openssl/bn.h>
That can be solved by explicitly providing paths to OpenSSL headers:
$ qmake INCLUDEPATH+="/usr/local/opt/openssl/include" ..
By the way, Homebrew had a warning about something similar:
firstname.lastname@example.org is keg-only, which means it was not symlinked into /usr/local, because macOS provides LibreSSL. If you need to have email@example.com first in your PATH, run: echo 'export PATH="/firstname.lastname@example.org/bin:$PATH"' >> /Users/YOURNAME/.bash_profile For compilers to find email@example.com you may need to set: export LDFLAGS="-Lfirstname.lastname@example.org/lib" export CPPFLAGS="-Iemail@example.com/include" For pkg-config to find firstname.lastname@example.org you may need to set: export PKG_CONFIG_PATH="/email@example.com/lib/pkgconfig"
and I tried to do so in my
export PATH="/usr/local/opt/openssl/bin:$PATH" export LDFLAGS="-L/usr/local/opt/openssl/lib" export CPPFLAGS="-I/usr/local/opt/openssl/include" export PKG_CONFIG_PATH="/usr/local/opt/openssl/lib/pkgconfig"
but that didn’t help. And providing paths via
qmake CLI parameters did help, so that’s what I will be doing going forward.
Next error I got was this:
../../../libretroshare/src/util/rsjson.h:24:10: fatal error: 'rapidjson/document.h' file not found #include <rapidjson/document.h>
That is solved by installing RapidJSON:
$ brew install rapidjson
Then there was this error:
../../../libretroshare/src/rsserver/p3face-info.cc:31:11: fatal error: 'sqlcipher/sqlite3.h' file not found # include <sqlcipher/sqlite3.h>
That is solved by installing SQLCipher:
$ brew install sqlcipher
Missing paths to linked libraries
After that everything finally built, but failed on linking stage:
ld: library not found for -lsqlcipher ld: library not found for -lminiupnpc ld: library not found for -lssl
That is because providing just include paths is not enough, you also need to provide paths to linked libraries binaries. So, the final sequence of commands to configure and build RetroShare (without plugins) is this:
$ cd /path/to/rapidshare/src $ mkdir build && cd $_ $ qmake \ INCLUDEPATH+="/usr/local/opt/openssl/include" QMAKE_LIBDIR+="/usr/local/opt/openssl/lib" \ QMAKE_LIBDIR+="/usr/local/opt/sqlcipher/lib" \ QMAKE_LIBDIR+="/usr/local/opt/miniupnpc/lib" \ .. $ make -j12
You might want to enable/disable certain application features:
- by default
autologinis disabled, so you’ll need to enter your password every time you launch RetroShare, which is certainly more secure but also rather annoying
- another example is
retroshare_service, which, if I understood correctly, allows RetroShare to run “headless” (without GUI). It is enabled by default, and I don’t need it
- not sure what
use_native_dialogsdoes, but it is disabled by default, and having enabled it I seem to get more responsive GUI dialogs in the application
So my configuration looks like this:
$ qmake \ INCLUDEPATH+="/usr/local/opt/openssl/include" QMAKE_LIBDIR+="/usr/local/opt/openssl/lib" \ QMAKE_LIBDIR+="/usr/local/opt/sqlcipher/lib" \ QMAKE_LIBDIR+="/usr/local/opt/miniupnpc/lib" \ CONFIG+=rs_autologin \ CONFIG+=no_retroshare_service \ CONFIG+=rs_use_native_dialogs \ CONFIG+=release \ ..
Building takes about 8 minutes on my Mac, and as a result I get a working RetroShare application in
RetroShare has some plugins as part of the main repository, and there are also plugins in separate repositories (for example, RetroChess).
To enable building RetroShare with plugins, you need to set this:
$ qmake CONFIG+=retroshare_plugins ..
If you want to build more plugins than just the ones from main repository, clone their repositories and copy sources to
/path/to/retroshare/src/plugins folder and add them in
SUBDIRS += \ VOIP \ FeedReader \ RetroChess # added
Before building you need to install required dependencies:
$ brew install ffmpeg $ brew install speex $ brew install speexdsp $ brew install opencv
The last one, OpenCV, is one crazy package. It drags a whole bunch of dependencies of its own, which in turn drag a lot more on their own, including by the way Qt. You can see the full list for yourself:
$ brew deps --tree --installed opencv
Anyway, having installed all the dependencies, you can now build the project. Needless to say, you’ll face errors about missing headers and linked libraries, so to save you some time, here are all of them:
$ qmake \ INCLUDEPATH+="/usr/local/opt/openssl/include" QMAKE_LIBDIR+="/usr/local/opt/openssl/lib" \ QMAKE_LIBDIR+="/usr/local/opt/sqlcipher/lib" \ QMAKE_LIBDIR+="/usr/local/opt/miniupnpc/lib" \ INCLUDEPATH+="/usr/local/opt/opencv/include/opencv4" QMAKE_LIBDIR+="/usr/local/opt/opencv/lib" \ INCLUDEPATH+="/usr/local/opt/speex/include" QMAKE_LIBDIR+="/usr/local/opt/speex/lib/" \ INCLUDEPATH+="/usr/local/opt/speexdsp/include" QMAKE_LIBDIR+="/usr/local/opt/speexdsp/lib/" \ QMAKE_LIBDIR+="/usr/local/opt/ffmpeg/lib" \ CONFIG+=retroshare_plugins \ ..
…Yet, you will still get errors, though different ones this time. Here’s the first one:
Undefined symbols for architecture x86_64: "cv::VideoCapture::VideoCapture(int, int)", referenced from: QVideoInputDevice::start() in QVideoDevice.o ld: symbol(s) not found for architecture x86_64 clang: error: linker command failed with exit code 1 (use -v to see invocation) make: *** [lib/libVOIP.dylib] Error 1 make: *** [sub-VOIP-make_first] Error 2 make: *** [plugins-make_first] Error 2
This one is happening because RetroShare doesn’t link to one of the libraries from OpenCV. To find out which one, you can google for
cv::VideoCapture and discover that it’s part of videoio, so you need to add that missing link with
Next error was more difficult to figure out:
usr/include/libxslt/xslt.h:83:22: error: default initialization of an object of const type 'const int'
Luckily, I’ve encountered something similar (not exactly) before, so soon enough I suspected that it’s yet again Mac OS providing an obsolete version of a library. And indeed, after I installed and used the latest XSLT library:
$ brew install libxslt
the problem was gone. So, here’s the final configuration command (with the features I’ve enabled):
$ qmake \ INCLUDEPATH+="/usr/local/opt/openssl/include" QMAKE_LIBDIR+="/usr/local/opt/openssl/lib" \ QMAKE_LIBDIR+="/usr/local/opt/sqlcipher/lib" \ QMAKE_LIBDIR+="/usr/local/opt/miniupnpc/lib" \ INCLUDEPATH+="/usr/local/opt/opencv/include/opencv4" QMAKE_LIBDIR+="/usr/local/opt/opencv/lib" \ INCLUDEPATH+="/usr/local/opt/speex/include" QMAKE_LIBDIR+="/usr/local/opt/speex/lib/" \ INCLUDEPATH+="/usr/local/opt/speexdsp/include" QMAKE_LIBDIR+="/usr/local/opt/speexdsp/lib/" \ INCLUDEPATH+="/usr/local/opt/libxslt/include" QMAKE_LIBDIR+="/usr/local/opt/libxslt/lib" \ QMAKE_LIBDIR+="/usr/local/opt/ffmpeg/lib" \ LIBS+=-lopencv_videoio \ CONFIG+=retroshare_plugins \ CONFIG+=rs_autologin \ CONFIG+=no_retroshare_service \ CONFIG+=rs_use_native_dialogs \ CONFIG+=release \ ..
Having built such a configuration, aside from
RetroShare.app you will also get plugins built in
/path/to/retroshare/src/build/plugins/**/lib/*.dylib. These files needs to be either bundled into
RetroShare.app/Contents/Resources/ or copied to
/Users/YOURNAME/.retroshare/extensions6/ (or whichever path is set in Preferences). Having done that, when you launch RetroShare next time, it should discover the plugins:
What about building from Qt Creator
You might ask, why building with bare
qmake from CLI, why not building from Qt Creator?
Well, first of all, building from CLI is just faster and actually more convenient. But also here’s what I get when I try to build the RetroShare project from Qt Creator:
/bin/sh: cmake: command not found make: *** [../../supportlibs/udp-discovery-cpp/libudp-discovery.a] Error 127
And I’ve no idea what this is about. I have CMake discoverable in my PATH, and also it is added in Qt Creator settings, and also I can build other CMake projects from Qt Creator just fine. Yeah, no idea.
There are certain styles files which need to be bundled in the application bundle. Not sure why those are not added to resources.
So, if you won’t bundle those, the application will still work, but you will be missing some skins/styles in the list of available style options, for example Bubble style for chats:
Bundling stuff into application bundle on Mac OS with
qmake is done via QMAKE_BUNDLE_DATA. In
/path/to/retroshare/src/retroshare-gui/src/retroshare-gui.pro on line 275 there is a block for
macx scope - that’s where the application icon is getting bundled, by the way. To bundle styles (or other resources) too, add the following lines:
dplQSS.files = $$PWD/qss dplQSS.path = Contents/Resources QMAKE_BUNDLE_DATA += dplQSS dplChatStyles.files = \ $$PWD/gui/qss/chat/Bubble \ $$PWD/gui/qss/chat/Bubble_Compact dplChatStyles.path = Contents/Resources/stylesheets QMAKE_BUNDLE_DATA += dplChatStyles
Having configured and built the application, you should get the following contents in your RetroShare application bundle:
$ tree -L 2 /path/to/retroshare/src/build/retroshare-gui/src/RetroShare.app/Contents/Resources/ ├── empty.lproj ├── qss │ ├── blacknight/ │ ├── blue/ │ ├── blue.qss │ ├── groove.qss │ ├── orangesurfer/ │ ├── orangesurfer.qss │ ├── qdarkstyle/ │ ├── qdarkstyle-v2.qss │ ├── qdarkstyle.qss │ ├── qlive/ │ ├── qlive.qss │ ├── redscorpion/ │ ├── redscorpion.qss │ ├── silver/ │ ├── silver.qss │ ├── uus/ │ ├── uus.qss │ ├── yaba/ │ ├── yaba.qss │ ├── yeah/ │ └── yeah.qss ├── rsMacIcon.icns └── stylesheets ├── Bubble/ └── Bubble_Compact/ 14 directories, 13 files
Making a DMG
If you’d like to distribute your build to other people, you’ll need to bundle required libraries and frameworks and also make a DMG.
Why DMG? No particular reason, really. It can just as well be a ZIP archive, but DMG seem to be more common for distributing applications on Mac OS.
In order to find and copy all the required libraries and frameworks, I would recommend to use macdeployqt. Actually, I would not recommend it, as it seem to bundle some redundant things as well, for example Virtual Keyboard, which I don’t think is used in RetroShare. But still - it does the job, so it’s easier to copy all the dependencies with
macdeployqt than to find and copy them manually. Though RetroShare plugins do need to be copied manually.
macdeployqt can also make a DMG. So here’s the entire chain of commands for deployment:
$ cd /path/to/retroshare/src/build $ cp \ ./plugins/FeedReader/lib/libFeedReader.dylib \ ./plugins/VOIP/lib/libVOIP.dylib \ ./plugins/RetroChess/lib/libRetroChess.dylib \ ./retroshare-gui/src/RetroShare.app/Contents/Resources/ $ du -hs ./retroshare-gui/src/RetroShare.app 28M $ macdeployqt ./retroshare-gui/src/RetroShare.app -dmg -verbose=2 $ du -hs ./retroshare-gui/src/RetroShare.app 220M $ du -hs ./retroshare-gui/src/RetroShare.dmg 81M
Homebrew is awesome
I must say, Homebrew is an absolute life savior. Getting (and building) all the dependencies without it would take dramatically more time. Or maybe I would have just given up halfway.
I’ve never used RetroShare before, and here’s what I can tell after using it for a week.
Documentation isn’t great
Documentation, and namely Mac OS build instructions, is scattered across several places. I could count at least five:
- One repository
- Another repository
- Three documents (and a patch) in the main repository:
The manual for building is not very helpful, which is why I created this very article. I would even say, the instructions from manual are half-baked and written in a negligent manner, there are a lot of typos and it even contains nonsense like:
In QtCreator Option Git add its path:
That is from the section about building RetroShare on Mac OS.
General documentation content also seems to be outdated. There are screenshots of menus and settings that no longer exist, and some instructions refer to no longer existing things as well.
Surely, I would be happy to make my contribution and update the documentation with my instructions, and also to fix typos and to make it look better. But given its current state, it just feels like my efforts would go into a void. At the very least documentation should be concentrated in one place first.
RetroShare requires you to have “friends” - other people’s nodes who added you as their “friend”. There are no servers: basically everyone is a server, and together they form a network.
That is the first difficulty you will encounter, but that is not an issue, and actually that is one of the main features of RetroShare. It’s even awesome. Also, luckily enough, I’ve got invites from several people, so I had at least something to connect to.
The real issues started to appear later: I tried running RetroShare on Windows, Mac OS and Linux, and I had different experience on all three.
On Windows I seemed to get connected at first, and I was even getting forums, chats and so on, but the very next day it just stopped: I am no longer able to see any forums, and all the chats I entered the day before now have no people in them.
The NAT and DHT indicators show the following:
And neither of them ever got green.
Then I tried it on Mac OS, as by that moment I’ve managed to build it. There it was better, and I get forum posts, chats and so on.
The NAT and DHT indicators, however, are all red:
Plus Network settings seem to have some troubles getting external IP, though I am not sure if it matters:
Finally, I tried RetroShare in a virtual machine running Linux, and would you look at that:
Everything is green!
No idea, how come the same thing works so differently across different platforms in the same network. I even tried forwarding ports on my router - that didn’t change a thing: still, only Linux had green DHT indicator, and Windows was still having troubles getting content.
As much as I wanted to like it, as a user, I can’t say that I will use RetroShare every day or rely on it for something important. It could be that this is because I am still an unexperienced user, but it has been more than a week as I’m trying to get familiar with it, and now I am about to give up.
The idea is certainly great, no doubt about that, especially looking at how “normal” internet is going to shit, but implementation is certainly not perfect. Aside from inconsistent connectivity, there are random (but seldom) application crashes, GUI can freeze on certain actions, notification flags on tabs do not disappear even though there are no unread items left, etc - all that results in not a great user experience.
On the bright side, though, the repository is rather active, so hopefully it all will get better eventually. Maybe I will even find some time to help fixing those issues.
And as a developer, I must admit the fact that (despite the length of my article) RetroShare actually builds just fine on Mac OS, and almost all the troubles I had were coming from missing dependencies. Given the cross-platform nature and the size of the project, this is quite impressive, especially considering that Mac OS is not its primary target platform.
Here are magnet links for my RetroShare 0.6.6 builds on Mac OS:
Let me know if it fails to run on your system, because I suspect that
macdeployqt doesn’t actually pack all the dependencies.
Zuck: Just ask
Zuck: I have over 4,000 emails, pictures, addresses, SNS
smb: What? How'd you manage that one?
Zuck: People just submitted it.
Zuck: I don't know why.
Zuck: They "trust me"
Zuck: Dumb fucks