Using Graphene to run applications inside SGX enclaves

Install dependencies (not comprehensive, contains only non-obvious):

$ pip3 install google-api-python-client

Get Graphene:

$ git clone https://github.com/oscarlab/graphene.git

Check the FSGSBASE support, which can be validated from AT_HW_CAP2:

$ LD_SHOW_AUXV=1 /bin/true | grep AT_HWCAP2
AT_HWCAP2:            0x2

Go to the cloned directory, and build Graphene:

$ cd graphene
$ ISGX_DRIVER_PATH= make SGX=1 DEBUG=1

Create a signing key for enclaves:

$ openssl genrsa -3 -out enclave-key.pem 3072
$ export SGX_SIGNER_KEY=$PWD/enclave-key.pem

There is an example, which runs bash inside an enclave. Let's give that a shot!

First, build it:

$ cd Examples/bash
$ make SGX=1 DEBUG=1

Then, you can run it:

$ SGX=1 ./pal_loader ./bash -c "ls"

Low-latency audio settings for Linux

I'll briefly describe how I go on configuring Linux for low-latency audio in Ubuntu and its derivatives.

Threaded interrupt handlers

When threaded interrupt handlers are enabled, kernel only acknowledges the triggered interrupt with preemption disabled, and right after that assigns a thread for the interrupt handler. The overall latency is reduced because user space processes can be scheduled almost immediately.

Threaded interrupt handlers can be enabled by adding threadirqs to GRUB_CMDLINE_LINUX in /etc/default/grub, and running update-grub. After tforfhe next reboot, they are activated.

rtirq-init package contains rtirq initialization script, which assigns priority to all interrupt handlers according to the rules in /etc/default/rtirq. The default configuration gives the highest priority for the sound card and USB host controller:

$ ps -T -o comm,policy,rtprio -p $(pgrep -w -d ',' irq) | egrep '(snd|hci)'
irq/126-xhci_hc FF      85
irq/143-snd_hda FF      90

This reduces audio latency as the sound card gets always served first.

CPU frequency scaling

In order to keep audio latency in a steady state, the CPU should always run at a constant frequency.

Downscaling

Downscaling occurs when the operating frequency of a CPU is decreased. The default power governor, powersave, does this when the demand for computing is low.

Downscaling can be disabled by using another power governor called performance, which keeps the CPU operating at its maximum frequency. A straight-forward way to enable it is to install cpufrequtils package, and create a file called /etc/default/cpufrequtils with the following statement:

GOVERNOR="performance"

After reboot the governor should have been changed:

$ cat /sys/devices/system/cpu/cpu*/cpufreq/scaling_governor
performance
performance
performance
performance
performance
performance

Upscaling

Upscaling occurs when the operating frequency of a CPU is temporarily increased above its maximum operating frequency. Intel CPU's have a feature called Turbo Boost, which causes the CPU to automatically upscale when the demand for processing is high.

Here's a systemd service disabling Turbo Boost, taken from a blog post:

$ cat /etc/systemd/system/disable-turbo-boost.service
[Unit]
Description=Disable Turbo Boost on a Intel CPU

[Service]
ExecStart=/bin/sh -c "echo 1 > /sys/devices/system/cpu/intel_pstate/no_turbo"
ExecStop=/bin/sh -c "echo 0 > /sys/devices/system/cpu/intel_pstate/no_turbo"
RemainAfterExit=yes

[Install]
WantedBy=sysinit.target

Resource limits

Popular audio software, such as JACK and Reaper, can usually take the advantage of real-time scheduling and memory locking, if such resources are available for the application process.

Resource limits for can be configured by creating a new config file to /etc/security/limits.d. The approach that I use is a supplemental group for low latency audio:

$ cat | sudo tee /etc/security/limits.d/audio.user.conf
@audio.user - memlock   8388608 # 8 GB
@audio.user - rtprio    40

JACK Audio Connection Kit (JACK)

It's not advisable to set memory locking unlimited, but rather set an appropriate fixed limit for your needs. This will provide some governance against software bugs, and also prevents user unknowingly overloading the system.

The priorities of user threads should be capped well below the interrupt handler priorities, as hardware should be served first. If this does not happen, it could at worst cause the hardware to fail.

JACK Audio Connection Kit, or just JACK, is a low-latency audio server for multiplexing the audio hardware for multiple clients. Its the most modern incarnation, JACK2, is centered around a command-line tool called jack_control, interfacing jackdbus daemon process, which manages the actual audio server jackd and PulseAudio interconnections with it.

Configuration

I've created ~/bin/jack_init script for reconfiguring JACK2:

#!/bin/sh

SOUND_CARD=$1
SAMPLE_RATE=$2
FRAMES_PER_INT=$3
FRAME_SIZE=$4

# Use the ALSA backend.
jack_control ds alsa

# Enable rtprios.
jack_control eps realtime true

# Configure the sound card.
jack_control dps device $SOUND_CARD
jack_control dps capture $SOUND_CARD
jack_control dps playback $SOUND_CARD
jack_control dps rate $SAMPLE_RATE
jack_control dps nperiods $FRAMES_PER_INT
jack_control dps period $FRAME_SIZE

For example, running jack_init hw:EVO4 3 256 populates ~/.config/jack/conf.xml with

<jack>
 <engine>
  <option name="driver">alsa</option>
  <option name="realtime">true</option>
 </engine>
 <drivers>  
  <driver name="alsa">
   <option name="device">hw:EVO4</option>
   <option name="capture">hw:EVO4</option>
   <option name="playback">hw:EVO4</option>
   <option name="rate">48000</option>
   <option name="period">256</option>
   <option name="nperiods">3</option>
  </driver>
 </drivers>
</jack>

The JACK run-time logs are stored to ~/.log/jack/jackdbus.log.

Sampling a web browser to Reaper using JACK

JACK gives quite convenient tools to sample audio from a web browser, or any other desktop application, which is sometimes so much convenient than trying to save it as a file. It's just a trivial matter of re-routing PulseAudio interconnections:

$ jack_connect "PulseAudio JACK Sink:front-left" REAPER:in1
$ jack_connect "PulseAudio JACK Sink:front-right" REAPER:in2
$ jack_disconnect "PulseAudio JACK Sink:front-left" system:playback_1
$ jack_disconnect "PulseAudio JACK Sink:front-right" system:playback_2

Now desktop audio can be only heard when monitored from Reaper and can be trivially record to any track.