How To Install Survloop on an Ubuntu 20.04 Server

Updated March 27, 2021 by Morgan Lesko

For now, this installation process has only been tested on a fresh DigitalOcean Droplet of any size with Ubuntu 20.04. Near the bottom of the Create Droplet page, you need to add an SSH Key for the root user — not a password. (The following instructions assume you are using the default SSH Key on your local computer.)

If you plan to associate your installation with a domain name, and want to install an SSL certificate below (with EFF's CertBot), then open up DigitalOcean's Networking tab now. Add your domain for DNS management, with A records for "@" and "www" pointing to your new Droplet's IP address.

It generally seems best to take advantage of DigitalOcean's Managed Databases. It's easiest to create one first, assign your Droplet as a Trusted Source, and create a new database for this installation. The auto-install scripts below assume an external database. The list of All Commands below does include instructions to install a MYSQL server within your new Droplet instead.


Create Super User, Disable Root

For these instructions, the new user is named rockhopuser, but you can call it anything.

Instead of 22, we will select a SSH Port to connect to (between 23 and 1023). The custom SSH port here will be 123.

For these instructions, the Server IP Address will be 123.45.67.890. We will setup SSH access restricted to your fixed IP Adress — hopefully this is your VPN's IP Address — here it will be 98.76.54.321.

% ssh root@server.ip.address
e.g.
% ssh root@123.45.67.890


Temporary Bug Workaround

Recently, fresh DigitalOcean Droplets have been giving errors when 'apt upgrade' is run. A post helped me through this [temporary] problem...

Before running the main installation steps below, run the first three commands manually:

$ apt-add-repository universe
$ apt update
$ echo "Y" | apt upgrade

If the 'apt upgrade' command hangs on a sshd_config screen, close your terminal tab and reconnect to the server in a fresh terminal. Then get a list of locked processed:

sudo lsof /var/cache/apt/archives/lock

Hopefully, this provides some results including a line like this:

focal   14627 root   66u   REG  259,3        0 12328392 /var/cache/apt/archives/lock

Use those results to identify the process number to kill:

sudo kill 14627  # focal upgrade process

If this workaround works, you should now hopefully be able to continue the installation process as intended below.



In your new server, first pull down a copy of the Survloop installation scripts. Then run the Create Super User script to create a super user to be used instead of the root account. This will restrict SSH access to your IP, or that of your VPN tunnel. This script includes an installation of Fail2ban, and disables some unneeded networking tools.

When prompted, enter your new super user's strong password — and copy it somewhere super duper safe, like a password manager. This first installation script will configure your SSH access to a custom port number. Be sure to securely copy this port number too.

If you have a YubiKey, then you can optionally enter your token for UFA. Press your YubiKey USB button, and delete all but the first 12 characters for this token. (SSH Public key+MFA with Yubikey on Ubuntu 20.04 LTS)


$ git clone http://github.com/rockhopsoft/install-scripts
$ bash install-scripts/src/ubuntu20/survloop/01-create-user.sh

This is a summary of Digital Ocean's Initial Server Setup with Ubuntu 20.04. The last line copies your SSH Key from your root user to your new one.

$ apt-add-repository universe
$ apt update
$ echo "Y" | apt upgrade
$ adduser rockhopuser
$ usermod -aG sudo rockhopuser
$ rsync --archive --chown=rockhopuser:rockhopuser ~/.ssh /home/rockhopuser


If you have a Yubikey...

You will need the first 12 characters of your Yubikey, e.g. cccccfghjcff:

$ apt install libpam-yubico -y
$ echo "rockhopuser:cccccfghjcff" >> /etc/yubico
$ sed -i 's/@include common-auth/auth required pam_yubico.so id=16 debug authfile=\/etc\/yubico/g' /etc/pam.d/sshd
$ sed -i 's/ChallengeResponseAuthentication no/ChallengeResponseAuthentication yes/g' /etc/ssh/sshd_config
$ sed -i 's/# Authentication:/AuthenticationMethods publickey,keyboard-interactive/g' /etc/ssh/sshd_config
$ sed -i 's/UsePAM no/UsePAM yes/g' /etc/ssh/sshd_config
$ systemctl restart sshd
If you do not have a Yubikey...
$ sed -i 's/UsePAM yes/UsePAM no/g' /etc/ssh/sshd_config
$ systemctl restart sshd


Edit Port and User IP in Uncomplicated Firewall (UFW)

Instead of 22, what SSH PORT will you connect to, between 23 and 1023? e.g. 123

This will also restrict SSH access to your VPN's IP, e.g. 98.76.54.321

$ sed -i "s/#Port 22/Port 123/g" /etc/ssh/sshd_config
$ sed -i 's/PermitRootLogin yes/PermitRootLogin no/g' /etc/ssh/sshd_config
$ sed -i 's/#LogLevel INFO/LogLevel VERBOSE/g' /etc/ssh/sshd_config
$ ufw default deny incoming
$ ufw default allow outgoing
$ ufw limit from 98.76.54.321 to any port 123
$ echo "y" | ufw enable
$ systemctl restart sshd

Disabling various over~networking

$ sed -i 's/#net.ipv4.conf.default.rp_filter=1/net.ipv4.conf.default.rp_filter=1/g' /etc/sysctl.conf
$ sed -i 's/#net.ipv4.conf.all.rp_filter=1/net.ipv4.conf.all.rp_filter=1/g' /etc/sysctl.conf
$ sed -i 's/#net.ipv4.conf.all.accept_redirects = 0/net.ipv4.conf.all.accept_redirects = 0/g' /etc/sysctl.conf
$ sed -i 's/#net.ipv6.conf.all.accept_redirects = 0/net.ipv6.conf.all.accept_redirects = 0/g' /etc/sysctl.conf
$ sed -i 's/#net.ipv4.conf.all.send_redirects = 0/net.ipv4.conf.all.send_redirects = 0/g' /etc/sysctl.conf
$ sed -i 's/#net.ipv4.conf.all.accept_source_route = 0/net.ipv4.conf.all.accept_source_route = 0/g' /etc/sysctl.conf
$ sed -i 's/#net.ipv6.conf.all.accept_source_route = 0/net.ipv6.conf.all.accept_source_route = 0/g' /etc/sysctl.conf
$ sed -i 's/#net.ipv4.conf.all.log_martians = 1/net.ipv4.conf.all.log_martians = 1/g' /etc/sysctl.conf
$ sysctl -p


Install Fail2ban

Include exception for your VPN's IP address:

$ apt update
$ apt upgrade
$ add-apt-repository universe
$ echo "Y" | apt install fail2ban
$ systemctl start fail2ban
$ systemctl enable fail2ban
$ cp /etc/fail2ban/jail.conf /etc/fail2ban/jail.local
$ sed -i "s/#ignoreip = 127.0.0.1\/8 ::1/ignoreip = 98.76.54.321/g" /etc/fail2ban/jail.local
$ sed -i 's/bantime  = 10m/bantime  = 30m/g' /etc/fail2ban/jail.local
$ sed -i 's/maxretry = 5/maxretry = 3/g' /etc/fail2ban/jail.local
$ sed -i 's/enabled = false/enabled = true/g' /etc/fail2ban/jail.local
$ systemctl enable fail2ban
$ systemctl status fail2ban.service
$ fail2ban-client status sshd


Give the server a minute or two to reboot...

# reboot

Then log back into the server with your new super user.

% ssh super_user@server.ip.address -p custom_ssh_port
e.g.
% ssh rockhopuser@123.45.67.890 -p 123


Associate A Domain Name

This is a good time to associate your domain with the server, which is described in Step 3 of the DigitalOcean tutorial.



Create A Database

Installing MYSQL Server

Only do this is you are not using another Managed Database, like the ones DigitalOcean offers. When you create a MYSQL password, be sure to copy and securely save it.

# apt install mysql-server
# mysql_secure_installation

During the MYSQL install, I choose to...

  • n - No thanks, password plugin
  • strong-password - Needs to be copied and saved somewhere safe
  • y - Remove anonymous user
  • y - Disallow root login remotely
  • y - Remove test database
  • y - Reload privelege tables

 

# mysql
mysql> ALTER USER 'root'@'localhost' IDENTIFIED WITH mysql_native_password BY 'super_strong_password';
mysql> FLUSH PRIVILEGES;
mysql> exit

Enable Remote MYSQL Connection (>=v5.7)

# nano /etc/mysql/mysql.conf.d/mysqld.cnf

Comment out the line for bind-address, set it to 0.0.0.0, or increase security with your own IP address.

You might want a GUI to play with your databases. I've enjoyed Sequel Pro.

Give your new SSH connection an appropriate name, and use your Droplet's IP address for the hosts. Copy the MYSQL root password you created while installing the core packages above.

Finally, link your SSH Key in your operating system, "~/.ssh/id_rsa.pub". This can sometimes be challenging since this is a hidden folder which you may need to work to reveal.

Once you are connected, use the drop-down in the top-left corner to "Add Database...". Give your database an appropriate name, and write it down for later.



Install Survloop

As your super user, enter 'sudo su' mode:

$ sudo su


Run the Survloop script built for Ubuntu 20.04:

# bash /root/install-scripts/src/ubuntu20/survloop/02-survloop.sh

Install LEMP Stack

This is a summary of How To Install Linux, Nginx, MySQL, PHP (LEMP stack) on Ubuntu 20.04

Run this series of commands, accepting defaults, and confirmations by pressing the Enter key...

# apt update
# echo "Y" | apt install nginx
# ufw allow 'Nginx HTTP'
# ufw allow 'Nginx HTTPS'
# echo "y" | ufw enable
# echo "Y" | apt install zip unzip php-fpm php-mysql php-mbstring php-xml php-bcmath php7.4-zip php7.4-gd ghostscript
# systemctl reload nginx



Setup Server's Domain

Swap out example.com with your domain, if you want to set one up now.

# nano /etc/nginx/sites-available/example.com

Add the following content, which was taken and slightly modified from the default server block configuration file, to your new server block configuration file (swap out domain):

server {
listen 80;
root /var/www/example.com/public;
server_name example.com www.example.com server.ip.address;
index index.php index.html index.htm index.nginx-debian.html;
# add_header X-Frame-Options "SAMEORIGIN";
add_header X-XSS-Protection "1; mode=block";
add_header X-Content-Type-Options "nosniff";

charset utf-8;
location / {
try_files $uri $uri/ /index.php?$query_string;
}

location = /favicon.ico { access_log off; log_not_found off; }
location = /robots.txt  { access_log off; log_not_found off; }

error_page 404 /index.php;

location ~ \.php$ {
fastcgi_pass unix:/var/run/php/php7.4-fpm.sock;
fastcgi_param SCRIPT_FILENAME $realpath_root$fastcgi_script_name;
include fastcgi_params;
}
location ~ /\.(?!well-known).* {
deny all;
}
}

You can read more about these settings in the original DigitalOcean instructions. Save and close the file. Create a link for it, and remove the default:

# ln -s /etc/nginx/sites-available/example.com /etc/nginx/sites-enabled/
# unlink /etc/nginx/sites-enabled/default

You can quickly double-check for any syntax errors by running this:

# sudo nginx -t
# sudo systemctl reload nginx


Test That PHP Is Running (optional)

Create a new file in the default public root:

# nano /var/www/example.com/public/info.php

Type "<\?php phpinfo();" into the empty file, and save it. If things are working, you should be able to type your IP address (or propagated domain name) into your browser, and see a bunch of details about your PHP installation, and save the file:

e.g. http://111.111.111.111/info.php


Create Swap File (optional)

If you use DigitalOcean's cheapest and smallest default Droplet, then you will need to create some more temporary memory to complete the install process. This method has worked for me:

# /bin/dd if=/dev/zero of=/var/swap.1 bs=1M count=1024
# /sbin/mkswap /var/swap.1
# /sbin/swapon /var/swap.1


Install Pear & Composer

Run this series of commands, accepting defaults, and confirmations by pressing the Enter key:

# echo "Y" | apt-get install php-pear pkg-config php-xml php7.4-xml php7.4-cli php-dev
# wget http://pear.php.net/go-pear.phar
# php go-pear.phar
# echo "Y" | apt-get install curl
# curl -sS https://getcomposer.org/installer | php -- --install-dir=/usr/local/bin --filename=composer


Install Laravel

Run this series of commands, accepting defaults, and confirmations by pressing the Enter key:

# rm -R /var/www/example.com
# composer create-project laravel/laravel /var/www/example.com 8.5.* --no-dev
# chown -R rockhopuser:rockhopuser /var/www/example.com
# cd /var/www/example.com
# composer update
# php artisan key:generate
# chown -R www-data:www-data storage bootstrap/cache resources/views database app/Models
# php artisan cache:clear
# composer require laravel/fortify
# composer update
# composer install --optimize-autoloader --no-dev
# nano .env

While editing Laravel's .env file, enter your domain name for the APP_URL, and database connection info:

APP_URL=https://example.com

DB_CONNECTION=mysql
DB_HOST=127.0.0.1
DB_PORT=3306
DB_DATABASE=newdatabasename
DB_USERNAME=root
DB_PASSWORD=strongpassword

Finally just clear the cache...

# php artisan config:clear
# COMPOSER_MEMORY_LIMIT=-1 composer require rockhopsoft/survloop
# nano composer.json

From your Laravel installation's root directory, update `composer.json` to require and easily reference Survloop:

...
"autoload": {
...
"psr-4": {
...
"RockHopSoft\\Survloop\\": "vendor/rockhopsoft/survloop/src/",
}
...
}, ...


It seems we also still need to manually edit `config/app.php`:

# nano config/app.php
...
'providers' => [
...
App\Providers\FortifyServiceProvider::class,
RockHopSoft\Survloop\SurvloopServiceProvider::class,
...
],
...
'aliases' => [
...
'Survloop' => 'RockHopSoft\Survloop\SurvloopFacade',
...
], ...

If installing on a server, you might also need to fix some permissions before the following steps...

# chown -R www-data:33 storage database app/Models

Clear caches and publish the package migrations...

# php artisan config:clear
# php artisan route:clear
# php artisan view:clear
# echo "0" | php artisan vendor:publish --force
# composer dump-autoload


Seed Survloop Database

With certain databases (like some managed by DigitalOcean), you may need to tweak the Laravel migration:

% nano database/migrations/2014_10_12_100000_create_password_resets_table.php
% nano database/migrations/2019_08_19_000000_create_failed_jobs_table.php

Add this line before the "Schema::create" line in each file:

\Illuminate\Support\Facades\DB::statement('SET SESSION sql_require_primary_key=0');

Then initialize the database:

$ php artisan migrate
$ php artisan db:seed --class=SurvloopSeeder
$ php artisan db:seed --class=ZipCodeSeeder
$ php artisan db:seed --class=ZipCodeSeeder2
$ php artisan db:seed --class=ZipCodeSeeder3
$ php artisan db:seed --class=ZipCodeSeeder4


Install SSL Certificate (Recommended)

If you plan to use a domain name, it is easiest to connect it to your server before installing the SSL Certificate.

Thanks to the EFF's Certbot, you can now get this step down quickly and easily... for free!! Just run these commands:

# apt-get update
# snap install --classic certbot
# certbot --nginx

Not sure this should be needed, but it helped my problems:

# ufw allow 'Nginx Full'
# ufw delete allow 'Nginx HTTP'
# nginx -t
# systemctl reload nginx
# ufw status verbose


Initialize Survloop Installation

Then browsing to the home page should prompt you to create the first admin user account:
http://survloop.local

If everything looks janky, then manually load the style sheets, etc:
http://survloop.local/css-reload

After logging in as an admin, this link rebuilds many supporting files:
http://survloop.local/dashboard/settings?refresh=2


Install Advanced Intrusion Detection Environment (Optional)

Advanced Intrusion Detection Environment (AIDE) might be worth installing too:

# echo "Y \n" | apt install aide
# aideinit
# cp /var/lib/aide/aide.db.new /var/lib/aide/aide.db
# cp /var/lib/aide/aide.conf.autogenerated /etc/aide/aide.conf



Frequently Update Ubuntu, etc

$ sudo apt-get update
$ sudo apt update && sudo apt dist-upgrade
$ sudo apt install update-manager-core
$ sudo do-release-upgrade
$ composer update
$ php artisan cache:clear

lastNode: 60 -> currNode: 60

coreID: -3,
POST/GET Requests:
Array
(
    [how-to-install-survloop-on-ubuntu-20_04] => 
)

Session:
Array
(
    [_token] => ERMWRmBVQTgFc66edbRULIQBaWLdpB9yl4ECbS3I
    [chkSysInit] => 1
    [_flash] => Array
        (
            [old] => Array
                (
                )

            [new] => Array
                (
                )

        )

)
sessLoops:
kidMap:
Array
(
)
1
loopItemIDs:
Array
(
)
1
dataSets:
Array
(
)
1

Page MicroLog

Time Elapsed

Start Page Load 0
Start loadAllSessData( 0
loadAllSessData( after loadSessionClear( 0
loadAllSessData( after loadSessInfo( 0
loadAllSessData( after loadSessionData( 0
loadAllSessData( after loadSessionDataSaves( 0
End loadAllSessData( after runLoopConditions( 0
TreeSurv index( 0
TreeSurv index( after survLoopInit 0
printTreePublic( start 0
Start TreeSurvLoad loadTree( 0
loadTree( after loadTreeStart( 0
loadTree( after loadTreeFromCache( 0
Start loadAllSessData( 0
loadAllSessData( after loadSessionClear( 0
loadAllSessData( after loadSessInfo( 0
loadAllSessData( after loadSessionData( 0
loadAllSessData( after loadSessionDataSaves( 0
End loadAllSessData( after runLoopConditions( 0
loadTree( after loadAllSessData( 0
printTreePublic( after loadTree() 0
printTreeGetCurrNode( after redirect checks 0
printTreeGetCurrNode( after loadAncestry( 0
printTreeGetCurrNode( start moving currNode 0
printTreeGetCurrNode( end moving currNode 0
printTreePublic( after multiRecordCheck( 0
printTreePublic( before printTreePublicCore( 0
An open source project by Morgan Lesko as
Rockhopper Software Designs logo
Survloop.org content is under
Creative Commons
2015-2021