I'm convinced that the lack of "SWAN_SYNC" in the Linux client is hindering this project.
In the near future (when faster GPUs will arrive) this performance loss will increase.
To prove it to you, I have set up an experiment consisting of two almost identical configurations:

That is a very nice comparison, and it certainly would be nice to give SWAN_SYNC in Linux a try.

But:

Note that until 29 July the Linux client had a GTX 1080Ti, which was about as fast as a GTX 980Ti in my Windows XP x64 host.

That is a very nice comparison, and it certainly would be nice to give SWAN_SYNC in Linux a try.

But:

Note that until 29 July the Linux client had a GTX 1080Ti, which was about as fast as a GTX 980Ti in my Windows XP x64 host.

This seems a little strange. The 1080Ti in Linux should easily beat the 980Ti in WinXP, unless it is somehow being limited. Maybe there is a memory or bus limitation of some sort?

It is probably the memory bus on the cards themselves that is the limitation. The GTX 980 Ti has 384 bits, while the GTX 1080 Ti has only 352 bits. While the memory on the 1080 Ti is faster, there is probably not much difference in bandwidth, which is probably what is limiting the performance of the 1080 Ti.

It is probably the memory bus on the cards themselves that is the limitation. The GTX 980 Ti has 384 bits, while the GTX 1080 Ti has only 352 bits. While the memory on the 1080 Ti is faster, there is probably not much difference in bandwidth, which is probably what is limiting the performance of the 1080 Ti.

That may explain the bandwidth, but it just confirms that the GTX 1080 Ti should be faster. I will be interested to see the numbers if they can enable SWAN_SYNC on Linux. Good luck in your effort to jog them on that.

Correct me if I'm wrong, but I think the main bottleneck for this application for operating systems without SWAN_SYNC is when the data comes over the PCIe BUS from the GPU, it is waiting in system memory for the CPU to do what it needs to do (I assume double precision compute) and send it back to the GPU for GPGPU compute.

With SWAN_SYNC it leaves one thread spun up so there is no delay in the data stream leading to a much more efficient process. As with Zoltan's data, you have up to 2000 seconds of time savings which on a WU is over 20% saved time AND power. The thing about processors is, even if they don't have data to compute, they still use almost the same amount of power when at max clock speed (which our GPUs are.)

With SWAN_SYNC it leaves one thread spun up so there is no delay in the data stream leading to a much more efficient process. As with Zoltan's data, you have up to 2000 seconds of time savings which on a WU is over 20% saved time AND power. The thing about processors is, even if they don't have data to compute, they still use almost the same amount of power when at max clock speed (which our GPUs are.)

Locking one thread will provide the least latency. Because the computation replies so much on the latency of the CPU, things like CPU clock speed, PCIe bandwidth, memory speed and operating system optimizations like SWAN_SYNC lower the latency far more than without SWAN_SYNC.

Latency is always the enemy.

Maybe. But that says that allowing the OS to chose the right core is worse than locking the GPU to a given core. That might be true when you have multiple CPU projects running. I am not sure it will show up in tests with all the cores free. So I think you will have to test it under varying conditions to see the real effect in practice.

Correct me if I'm wrong, but I think the main bottleneck for this application for operating systems without SWAN_SYNC is when the data comes over the PCIe BUS from the GPU, it is waiting in system memory

... for the CPU to do what it needs to do (I assume double precision compute)

and send it back to the GPU for GPGPU compute.

With SWAN_SYNC it leaves one thread spun up so there is no delay in the data stream leading to a much more efficient process.

I don't have very fast GPUs on at the moment, only a couple of GTX 750 Ti's. But they show 95 to 96% utilization under Linux. I will try out a GTX 1070 if they get SWAN_SYNC enabled, to see more of a difference. It looks promising, but it has been ages since I used it last under Windows.

I've compared the run times of my GTX 1080Tis under different OS.
My host Windows 10 / SWAN_SYNC on is faster by ~2.8% and ~8.6% than my hosts with Linux / SWAN_SYNC off.
Strangely the older Linux host (i3-4360) is faster and takes less CPU time too to process the GPUGrid workunits (the other host has an i3-7300).
BTW the Windows 10 host runs a Rosetta@home CPU task simultaneously, while the Linux hosts run only GPUGrid.
Host 113852: Win10, i7-4930K @ 4.4GHz, 16GB DDR3@1866MHz
Host 391255: Linux (Ubuntu 14.04 LTS), i3-4360 @ 3.7GHz, 4GB DDR3 @ 1333MHz CL6
Host 482653: Linux (Ubuntu 14.04 LTS), i3-7300 @ 4.0GHz, 32GB DDR4 @ 2133MHz

Very interesting thread. I don't have any answers but an observation. At first I thought those CPUs are different. Linux rig has an i7-4790k with 16 lanes, 4 GHz & 8 MB cache. WinXP rig has an i3-4160 with 16 lanes, 3.6 GHz & 4 MB cache. I've seen where MIP WUs at WCG choke for lack of L3 cache but here smaller cache computer is faster.
I've also seen much discussion of PCIe bus speed at F@H forum. These MBs are very different and the Asus manual is very uninformative. As I understand it, it takes 16 lanes to run the PCIe bus at full 3.0 x16 speed. The Asus manual says 16x and first 1x slots are shared. I assume nothing was plugged into either of the 1x slots. The Asus manual has no block diagram so maybe they use all 16 CPU lanes to run the single 16x slot and do all other bus functions with their B85 chip. Maybe not. GPU-Z can monitor the bus speed on a Windows computer but I don't recall seeing that here.
The GA-Z87X may be the issue. A much better manual with a block diagram shows that only one of the four 16x slots is capable of 3.0 x16 speed and that's if nothing else is plugged in. The Z87 handles the 1x slots and other bus functions. Is there a way with Linux to monitor actual bus speed???
Also, the BIOS usually has a version of an option for the PCIe speed: Auto, Gen 1, Gen 2 or Gen 3. I set all mine to Gen 3 from the default Auto.

I've compared the run times of my GTX 1080Tis under different OS.
My host Windows 10 / SWAN_SYNC on is faster by ~2.8% and ~8.6% than my hosts with Linux / SWAN_SYNC off.

Well it made a small difference.

With swan_sync, the PABLO_2IDP took 7 hours 13 minutes, and used 99.5% of the CPU.
Without swan_sync, the PABLO_2IDP took 7 hours 40 minutes, but used 60% of the CPU.

I would rather have the CPU. And I don't think drivers would make a difference.

... I assume nothing was plugged into either of the 1x slots.

The Asus manual has no block diagram so maybe they use all 16 CPU lanes to run the single 16x slot and do all other bus functions with their B85 chip.

The GA-Z87X may be the issue. A much better manual with a block diagram shows that only one of the four 16x slots is capable of 3.0 x16 speed and that's if nothing else is plugged in.

The Z87 handles the 1x slots and other bus functions. Is there a way with Linux to monitor actual bus speed???

Also, the BIOS usually has a version of an option for the PCIe speed: Auto, Gen 1, Gen 2 or Gen 3. I set all mine to Gen 3 from the default Auto.

What is the current CUDA version I should be using?

Well it made a small difference.

With swan_sync, the PABLO_2IDP took 7 hours 13 minutes, and used 99.5% of the CPU.
Without swan_sync, the PABLO_2IDP took 7 hours 40 minutes, but used 60% of the CPU.

I would rather have the CPU.

And I don't think drivers would make a difference.

Yes, I am sure it is useful. It would just take a faster card than I use at the moment.

To answer my on question:
Under Ubuntu 16.04 and 18.04, with the BOINC manager installed from the repository, the manager runs as a daemon, and it has it's own environment, so the SWAN_SYNC=1 setting should placed in it's own configuration file at

Hey Zoltan, what do you mean by edited as root? Does this mean you change User=boinc to User=root?

Hey Zoltan, what do you mean by edited as root? Does this mean you change User=boinc to User=root?

Good to hear that Zoltan and Rod4X4 worked it out how to get SWAN_SYNC=1 working on repository versions of BOINC. My posts in the Linux thread seems to have provided the necessary dialog and testing for the solution to be exposed.

So now both the Seti Tbar and repository versions of BOINC can benefit from the improvement in using SWAN_SYNC. Project benefits from increased throughput.

Hey Zoltan, what do you mean by edited as root? Does this mean you change User=boinc to User=root?

Try running the text editor as root; for example:
sudo gedit /lib/systemd/system/boinc-client.service

/lib/systemd/system/boinc-client.service
This file should be edited as 'root'.
In the [service] section of that file there should be a line containing: Environment="SWAN_SYNC=1"

First of all, thank you very much to all kind contributors to bringing this to work, in this and other threads.

Now, a step-by-step practical way to apply this in a few minutes (for Linux rookies like me and above)
I've just applied successfully this method in three different computers with already running GPU tasks without the need for them to end, and all tasks have restarted normally after applying the new configuration.
All three computers are running under Linux 16.04.5 LTS and repository BOINC versions (7.6.31 x64).

-1) If there are tasks running: Enter BOINC Manager - Activity - Suspend. Exit Boinc Manager. This will checkpoint and stop all running tasks
-2) Open a Terminal window, and execute the following commands:
-3) sudo killall boinc
-4) sudo gedit /lib/systemd/system/boinc-client.service
-5) Add the following line immediately below [Service] tag: Environment="SWAN_SYNC=1"
-6) Save the change and close gedit instance. Return to Terminal window
-7) sudo systemctl daemon-reload
-8) sudo /etc/init.d/boinc-client restart
-9) Close Terminal window
-10) Enter BOINC Manager - Activity - Run

That's all. Previous running tasks will restart from its stored checkpoint, but a full CPU thread will be assigned to every GPU.

One last advise:
Please, be caraful. This configuration will bring the whole system to its maximum performance.
More power = more heat. It Might be a good idea to check system fans and heatsinks to be free of dust...
And I use Psensor program from Linux repository. I recommend it to check temperatures, fan levels, and several other interesting indicatives.

Thank you all!

my gtx 1080 GPU is now on 99% usage with linux!
____________
Cruncher/Learner in progress.