Wednesday, May 23, 2018

GDPR or AVG - regain control Part 1: Your Own Cloud

Create your own Cloud

Replace Google or Dropbox, and gain control over you own data. Encrypt it, protect it, share your data only with who you want.

ARM, Ubuntu and secured Nextcloud

This episode will be followed by an entry on email. For now, I settle for a relatively cheap ARM device (an ODroid XU4, to be precise), run it with Ubuntu, and install NextCloud.
The choice for Ubuntu has everything to do with the availability of software for Ubuntu on ARM devices, therwise it would have been Debian.

Of course, you will need some sort of storage, which I do not include in the equasion here. Plenty of options, even by the same manufacturer.

Hardware considerations

The basic setup involved buying the device. You might want to opt for the newer 64-bit Single Board Computers (SBC), like the RaspBerry Pi 3, PINE ROCK64, or the ODROID C2, but I have chosen the XU4, with a 32-bit processor, but 8 cores, in stead of 4. This benefits the performance over products like the C2, because I can run more threads. The penalty comes in the form of memory addressing space, which is not a real problem, unless you want to upload files greater than 2GB...

Anyway, costs involved here are about USD 80 at the time of writing. Not only do you need the device it self, it needs a power supply, an eMMC memory module (just do it - performance is sooo much better than a micro SD card), an eMMC adapter and a case would be nice, too. I currently own a PINE Rock64, an ODroid C2 and XU4, and a Nanopi 2 Fire and got my XU4 with all components mentioned from ameriDroid.com, because they had the XU4 on stock. Very speedy delivery, and a personal "Thank you Frank" on the receipt by Renate & Amanda - how nice!

Basic setup

So, you have your device delivered, unpack it, hook it up to you screen, using the HDMI connector, and attach a USB keyboard. Fire it up, and see if the image of your choice (I ordered an Ubuntu eMMC image) boots correctly. If so, power down and remove the eMMC card. Connect it to the eMMC adapter, and insert the adapter in an SD slot on your (presumably, MS Windows) PC.

Download Etcher, the excellent image burner. Download a Minimal Ubuntu Xenial (16.04) version. Locations where to download from can be found on the Wiki page. I have chosen the ubuntu-16.04.3-4.14-minimal-odroid-xu4-20171213.img.xz.

Use Echter to burn the image (and Ether will unpack it, too!) to the eMMC (or SD) card. Move the card back to the XU4, and power up. It will boot, and spin down. This is normal! Just wait a minute, and power up again - it performed a resize of the root filesystem to max out the usage of your eMMC/SD card. Log in as root, with odroid as password. Change the password:
root@odroid:~# passwd

Let's get cracking: security!

You have now chosen a good, strong password for the root account. Good. BTW, strong does not mean you have use numbers and signs, it means you have a password with high entropy. Please note: a password, consisting of 12 upper and lower case characters has an entropy of 8 byte. Using 8 random characters (All ASCII printable) is just over 5 byte. Using an 8 character password with upper and lower case, numbers and signs (which lots of systems use...) is actually LESS SECURE than using 12 letter passwords. Please take a look at this...

Time to loose passwords! (but that's another blog entry)

Still as root, perform the following:
root@odroid:~# apt update root@odroid:~# apt upgrade root@odroid:~# apt dist-upgrade root@odroid:~# apt install linux-image-xu3
... (xu3! yes, that is not a typo. Answer No to the Abort Now? question) add to /etc/fstab:
tmpfs /run/shm tmpfs defaults,noexec,nosuid 0 0
Now, it is time to reboot with the latest kernel and programs:
root@odroid:~# shutdown -r now

Security: enable logins over SSH

I want to be able to remotely administer my devices, so I want SSH running. First of all, check on the versions. Most of the SBC images are created by enthousiasts, but maintaining images for all different SBC's is a hell of a job, so not all software will be at the last stand.
So, check the current version of SSL software, and get the latest:
root@odroid:~# openssl version --> 1.0.2g mar 2016 (!) root@odroid:~# cd /usr/local/src && apt install gcc make -y root@odroid:~# wget https://www.openssl.org/source/openssl-1.1.0h.tar.gz && tar xzvf openssl-1.1.0h.tar.gz && cd openssl-1.1.0h root@odroid:~# ./config -Wl,--enable-new-dtags,-rpath,'$(LIBRPATH)' root@odroid:~# make root@odroid:~# make install root@odroid:~# shutdown -r now root@odroid:~# opeenssl version --> OpenSSL 1.1.0h 27 Mar 2018

Security: enable network time and date

As many security mechanisms work with data, which is only valid for a limited time frame, having the correct time is crucial (at least, for the client and the server). Those little SBCs I have, do not all have battery backed up Real Time Clocks (RTCs) like AMD/Intel computer motherboards. There is a successor of the Network Time Protocol, timedatectl, which also handles calls of programs to RTCs.
root@odroid:~# vi /etc/systemd/timesyncd.conf [Time] # choose ntp servers geographically close to you NTP=3.nl.pool.ntp.org 0.nl.pool.ntp.org 2.nl.pool.ntp.org 1.nl.pool.ntp.org FallbackNTP=0.debian.pool.ntp.org 0.pool.ntp.org 1.pool.ntp.org 0.nl.pool.ntp.org # check/correct timezone root@odroid:~# dpkg-reconfigure tzdata # Check with: root@odroid:~# timedatectl Local time: Tue 2018-04-24 18:11:43 CEST Universal time: Tue 2018-04-24 16:11:43 UTC RTC time: Tue 2018-04-24 16:11:44 Time zone: Europe/Amsterdam (CEST, +0200) Network time on: yes NTP synchronized: yes RTC in local TZ: no

Security: network and host

Now, it is time to set up network and hostname. As this is a server, I want fixed hostname and IP-addresses, as well as some security measures... First of all: rework /etc/sysctl.conf:
# Uncomment the next two lines to enable Spoof protection (reverse-path filter) # Turn on Source Address Verification in all interfaces to # prevent some spoofing attacks net.ipv4.conf.default.rp_filter=1 net.ipv4.conf.all.rp_filter=1 # Uncomment the next line to enable TCP/IP SYN cookies # See http://lwn.net/Articles/277146/ # Note: This may impact IPv6 TCP sessions too net.ipv4.tcp_syncookies=1 net.ipv4.tcp_max_syn_backlog = 2048 net.ipv4.tcp_synack_retries = 2 net.ipv4.tcp_syn_retries = 5 # Do not accept ICMP redirects (prevent MITM attacks) net.ipv4.conf.all.accept_redirects = 0 net.ipv6.conf.all.accept_redirects = 0 # Do not send ICMP redirects (we are not a router) net.ipv4.conf.all.send_redirects = 0 # Do not accept IP source route packets (we are not a router) net.ipv4.conf.all.accept_source_route = 0 net.ipv6.conf.all.accept_source_route = 0 # # Log Martian Packets net.ipv4.conf.all.log_martians = 1 net.ipv4.icmp_ignore_bogus_error_responses = 1 #
Enable IP Spoofing protection, edit /etc/host.conf:
order bind,hosts # multi on nospoof on
Set the hostname:
hostnamectl set-hostname --static cloud
Set a fixed IP-address, disable NetworkManager, fix resolv.conf:
vi /etc/network/interfaces.d/eth0 allow-hotplug eth0 iface eth0 inet static address 192.168.4.220/24 gateway 192.168.4.254 name-server 192.168.4.200 # IPv6 is DHCP, and handled by the router iface eth0 inet6 dhcp root@odroid:~# service network-manager stop root@odroid:~# systemctl disable network-manager vi /etc/resolvconf/resolv.conf.d/head # router insists on adding itself, in stead of my PiHole. # add IPv4 and IPv6 addresses of your own nameserver(s) nameserver 192.168.4.200 shutdown -r now ## # reboot and see if everything still works, then: # sudo apt purge network-manager

Security: add user(s)

For every day maintenance, I use a normal account, that has the possibility for elevated commands (Super User do, or Substitute User do: sudo). Add it using:
adduser frank usermod -G sudo,ssh,operator frank id frank uid=1000(frank) gid=1000(frank) groups=1000(frank),27(sudo),37(operator),111(ssh)
I can now login using ssh frank@192.168.4.220. Check if everything is OK: try if you can use sudo.

Security: clean up

The less programs you have, the less security issues you will encounter. I found some stuff, installed by default, that I do not want/use: - beforementioned network manager - x11 (the GUI; this is a server, no GUI required!) So:
frank@cloud:~$sudo apt purge network-manager frank@cloud:~$sudo apt purge x11-common frank@cloud:~$sudo apt autoremove
I also do not want to be greeted by Ubuntu spam. Change the file /etc/pam.d/sshd from:
# Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. session optional pam_motd.so motd=/run/motd.dynamic session optional pam_motd.so noupdate # change to: # Print the message of the day upon successful login. # This includes a dynamically generated part from /run/motd.dynamic # and a static (admin-editable) part from /etc/motd. #session optional pam_motd.so motd=/run/motd.dynamic #session optional pam_motd.so noupdate
Install something different, and less inviting:
frank@cloud:~$ sudo apt install sysvbanner frank@cloud:~$ sudo vi /etc/issue.net Ongeautoriseerde toegang verboden! Unauthorized access forbidden!

Security: enhance encryption

In stead of 1024 bits, I use 4096 bits server keys and Diffie-Hellman parameters. Adapt /etc/sshd_config:
# HostKeys for protocol version 2 HostKey /etc/ssh/ssh_host_rsa_key #HostKey /etc/ssh/ssh_host_dsa_key #HostKey /etc/ssh/ssh_host_ecdsa_key #HostKey /etc/ssh/ssh_host_ed25519_key ... # Lifetime and size of ephemeral version 1 server key KeyRegenerationInterval 3600 ServerKeyBits 4096
Generate the Diffie-Hellman parameters (and take a break while you at it - this takes a long time!), and generate a new server key of 4096 bits:
sudo openssl dhparam -out /etc/ssl/certs/dhparam.pem 4096 sudo ssh-keygen -q -f /etc/ssh/ssh_host_rsa_key -N '' -b 4096 -t rsa

Security: no more password-based logins

Generate a key pair. I use MobaXterm, which can generate keys as well. Copy the public key to the server, in a special file. Make sure you are logged in as your daily maintenace user:
frank@cloud:~$ mkdir ~/.ssh frank@cloud:~$ vi ~/.ssh/authorized_keys # paste your public key, and save the file (ESC, :wq, ENTER) frank@cloud:~$ chmod 700 ~/.ssh frank@cloud:~$ chmod 600 ~/.ssh/authorized_keys
Now, log off, alter your terminal emulator to use your private key for logins, and log in again. As soon as you see 'Authenticating with public key "[any comment you gave when creating the key]"', success! You should now alter /etc/ssh/sshd_config, and alter, add, verify:
RSAAuthentication yes PubkeyAuthentication yes AuthorizedKeysFile %h/.ssh/authorized_keys PasswordAuthentication no
Now, reboot, and try logging in again. If you still see the message with Authenticating with key, it is time to disallow root logins alltogether!

Security: disable root logins

In order to disallow password-based and root logins completely, open /etc/ssh/sshd_config, and alter/add/verify:
PermitRootLogin no PermitEmptyPasswords no ChallengeResponseAuthentication no PasswordAuthentication no UsePAM no # Added AuthenticationMethods publickey Protocol 2 HostKey /etc/ssh/ssh_host_rsa_key ServerKeyBits 4096 SyslogFacility AUTH LogLevel INFO LoginGraceTime 2m StrictModes yes RSAAuthentication yes PubkeyAuthentication yes AuthorizedKeysFile %h/.ssh/authorized_keys
Reboot, and try root logins, and password based logins. You should encounter:
Ongeautoriseerde toegang verboden! Unauthorized access forbidden! Permission denied (publickey).

Let's get installing: nginx with cache!

Goal of it all is to get nginx, a high performance, low memory footprint http server running with NextCloud. I also want lots of caching, as these SBCs are not speedmonsters like the latest AMD or Intel chips. But they are very low power.
Unfortunately, nginx does not come with the cache purge module, so I will have to do that by hand. Log in, switch to a more permanent elevated session, and start installing and building(!) required software:
sudo -s cd /usr/local/src apt update && apt upgrade -y && apt install software-properties-common python-software-properties zip unzip screen curl ffmpeg libfile-fcntllock-perl -y wget http://nginx.org/keys/nginx_signing.key && apt-key add nginx_signing.key apt install language-pack-en-base -y && sudo LC_ALL=en_US.UTF-8 apt autoremove # In case anything might be installed: remove it completely apt remove nginx nginx-common nginx-full -y --allow-change-held-packages vi /etc/apt/sources.list.d/nginx.list # add: deb http://nginx.org/packages/ubuntu/ xenial nginx deb-src http://nginx.org/packages/ubuntu/ xenial nginx apt update
Now, in my case, this command ended with an error:
Err:3 http://nginx.org/packages/ubuntu xenial InRelease The following signatures couldn't be verified because the public key is not available: NO_PUBKEY ABF5BD827BD9BF62
The remedy is to install the nginx repository signing key:
wget http://nginx.org/keys/nginx_signing.key apt-key add ./nginx_signing.key apt update
Now it's ok. The following is to be expected, and can be ignored:
N: Skipping acquire of configured file 'nginx/binary-armhf/Packages' as repository 'http://nginx.org/packages/ubuntu xenial InRelease' doesn't support architecture 'armhf'


We will build it...
apt build-dep nginx -y # should not issue warnings apt source nginx
Errors to be ignored:
gpgv: Can't check signature: public key not found dpkg-source: warning: failed to verify signature on ./nginx_1.14.0-1~xenial.dsc W: Can't drop privileges for downloading as file 'nginx_1.14.0-1~xenial.dsc' couldn't be accessed by user '_apt'. - pkgAcquire::Run (13: Permission denied)
Just go ahead and proceed with:
mkdir -p /usr/local/src/nginx-1.14.0/debian/modules cd /usr/local/src/nginx-1.14.0/debian/modules ### ### THIS IS WHERE WE ADD STUFF TO NGINX ### wget https://github.com/FRiCKLE/ngx_cache_purge/archive/2.3.tar.gz tar -zxvf 2.3.tar.gz && rm 2.3.tar.gz cd .. vi rules # Find with-ld-opt="$(LDFLAGS): [ESC] /with-ld-opt="$(LDFLAGS) after the string, add: --add-module="$(CURDIR)/debian/modules/ngx_cache_purge-2.3" # Find: dh_shlibdeps -a append --dpkg-shlibdeps-params=--ignore-missing-info # save, change to the top directory and create the mew nginx: cd /usr/local/src/nginx-1.14.0 dpkg-buildpackage -uc -b -j4 cd /usr/local/src dpkg --install nginx_1.14.0-1~xenial_armhf.deb # Prevent nginx from being updated automagically: apt-mark hold nginx # Enable nginx service: systemctl enable nginx.service apt-mark hold nginx && systemctl enable nginx.service # alter the config in /etc/nginx/nginx.conf - only changes from the original are listed: 1,2c1,3 < user www-data; < worker_processes auto; --- > > user nginx; > worker_processes 1; 10,11d10 < multi_accept on; < use epoll; 16,19d14 < server_names_hash_bucket_size 64; < upstream php-handler { < server unix:/run/php/php7.2-fpm.sock; < } 21,24d15 < #include /etc/nginx/proxy.conf; < #include /etc/nginx/ssl.conf; < #include /etc/nginx/header.conf; < #include /etc/nginx/optimization.conf; 27,34c18,20 < log_format main '$remote_addr - $remote_user [$time_local] "$request" ' < '$status $body_bytes_sent "$http_referer" ' < '"$http_user_agent" "$http_x_forwarded_for"' < '"$host" sn="$server_name" ' < 'rt=$request_time ' < 'ua="$upstream_addr" us="$upstream_status" ' < 'ut="$upstream_response_time" ul="$upstream_response_length" ' < 'cs=$upstream_cache_status' ; --- > log_format main '$remote_addr - $remote_user [$time_local] "$request" ' > '$status $body_bytes_sent "$http_referer" ' > '"$http_user_agent" "$http_x_forwarded_for"'; 39,43c25,26 < send_timeout 3600; < tcp_nopush on; < tcp_nodelay on; < open_file_cache max=500 inactive=10m; < open_file_cache_errors on; --- > #tcp_nopush on; > 45,50c28,29 < reset_timedout_connection on; < server_tokens off; < resolver 192.168.4.200; < # resolver IP is your name server or router (e.g. your FritzBox) < resolver_timeout 10s;
Now, I have my storage server set up as NFS server, so I need the NFS client, and a permanent, boot resilient entry in the mount file, /etc/fstab:
# install nfs: apt install nfs-client # add to /etc/fstab: 192.168.4.140:/mnt/tank1/ds1/nc /var/nc nfs soft,intr,nosuid 0 0
You will have to work out your own options, like installing the samba client, and use a SMB share (MS Windows share), use local storage or yet another option. With large, 10 to 14TB disks or 2TB SSDs of date you still have a lot of local storage!.

NextCloud needs some directories, so create these:
mkdir -p /var/nc /var/www/letsencrypt /usr/local/tmp/cache /usr/local/tmp/sessions /usr/local/tmp/apc /upload_tmp chown -R www-data:www-data /upload_tmp /var/nc /var/www chown -R www-data:root /usr/local/tmp/sessions /usr/local/tmp/cache /usr/local/tmp/apc
Now, it is time to see what's cooking; test your nginx configuration, and correct any errors. If all is well, you should be able to surf to your localhost, and see the standard nginx welcome page.
# test the nginx configuration: nginx -t # When OK, start and see if purge is compiled within - one should get the ngx_cache_purge response: service nginx start nginx -V 2>&1 | grep ngx_cache_purge -o ngx_cache_purge # You could now do and see the nginx welcome message in html code: curl localhost

Installing and configuring: PHP 7.2

Nextcloud heavily depends on PHP, so I want the latest and the safest: version 7.2. Carsten Rieger has written the stream editor (sed) commands; I merely copied these off his site.
Let's get this show on the road:
apt install language-pack-en-base -y && sudo LC_ALL=en_US.UTF-8 add-apt-repository ppa:ondrej/php -y && apt update apt install php7.2-fpm php7.2-zip php7.2-mysql php7.2-curl php7.2-xml php7.2-intl php7.2-gd php7.2-mbstring php7.2-json php7.2-bz2 php7.2-ldap php-apcu imagemagick php-imagick -y # Thanks to Carsten Rieger: cp /etc/php/7.2/fpm/pool.d/www.conf /etc/php/7.2/fpm/pool.d/www.conf.bak cp /etc/php/7.2/cli/php.ini /etc/php/7.2/cli/php.ini.bak cp /etc/php/7.2/fpm/php.ini /etc/php/7.2/fpm/php.ini.bak cp /etc/php/7.2/fpm/php-fpm.conf /etc/php/7.2/fpm/php-fpm.conf.bak sed -i "s/;env\[HOSTNAME\] = /env[HOSTNAME] = /" /etc/php/7.2/fpm/pool.d/www.conf sed -i "s/;env\[TMP\] = /env[TMP] = /" /etc/php/7.2/fpm/pool.d/www.conf sed -i "s/;env\[TMPDIR\] = /env[TMPDIR] = /" /etc/php/7.2/fpm/pool.d/www.conf sed -i "s/;env\[TEMP\] = /env[TEMP] = /" /etc/php/7.2/fpm/pool.d/www.conf sed -i "s/;env\[PATH\] = /env[PATH] = /" /etc/php/7.2/fpm/pool.d/www.conf sed -i "s/pm.max_children = .*/pm.max_children = 240/" /etc/php/7.2/fpm/pool.d/www.conf sed -i "s/pm.start_servers = .*/pm.start_servers = 20/" /etc/php/7.2/fpm/pool.d/www.conf sed -i "s/pm.min_spare_servers = .*/pm.min_spare_servers = 10/" /etc/php/7.2/fpm/pool.d/www.conf sed -i "s/pm.max_spare_servers = .*/pm.max_spare_servers = 20/" /etc/php/7.2/fpm/pool.d/www.conf sed -i "s/;pm.max_requests = 500/pm.max_requests = 500/" /etc/php/7.2/fpm/pool.d/www.conf sed -i "s/output_buffering =.*/output_buffering = 'Off'/" /etc/php/7.2/cli/php.ini sed -i "s/max_execution_time =.*/max_execution_time = 1800/" /etc/php/7.2/cli/php.ini sed -i "s/max_input_time =.*/max_input_time = 3600/" /etc/php/7.2/cli/php.ini sed -i "s/post_max_size =.*/post_max_size = 10240M/" /etc/php/7.2/cli/php.ini sed -i "s/;upload_tmp_dir =.*/upload_tmp_dir = \/upload_tmp/" /etc/php/7.2/cli/php.ini sed -i "s/upload_max_filesize =.*/upload_max_filesize = 10240M/" /etc/php/7.2/cli/php.ini sed -i "s/max_file_uploads =.*/max_file_uploads = 100/" /etc/php/7.2/cli/php.ini sed -i "s/;date.timezone.*/date.timezone = Europe\/\Berlin/" /etc/php/7.2/cli/php.ini sed -i "s/;session.cookie_secure.*/session.cookie_secure = True/" /etc/php/7.2/cli/php.ini sed -i "s/;session.save_path =.*/session.save_path = \"N;700;\/usr\/local\/tmp\/sessions\"/" /etc/php/7.2/cli/php.ini sed -i '$aapc.enable_cli = 1' /etc/php/7.2/cli/php.ini sed -i "s/memory_limit = 128M/memory_limit = 512M/" /etc/php/7.2/fpm/php.ini sed -i "s/output_buffering =.*/output_buffering = 'Off'/" /etc/php/7.2/fpm/php.ini sed -i "s/max_execution_time =.*/max_execution_time = 1800/" /etc/php/7.2/fpm/php.ini sed -i "s/max_input_time =.*/max_input_time = 3600/" /etc/php/7.2/fpm/php.ini sed -i "s/post_max_size =.*/post_max_size = 10240M/" /etc/php/7.2/fpm/php.ini sed -i "s/;upload_tmp_dir =.*/upload_tmp_dir = \/upload_tmp/" /etc/php/7.2/fpm/php.ini sed -i "s/upload_max_filesize =.*/upload_max_filesize = 10240M/" /etc/php/7.2/fpm/php.ini sed -i "s/max_file_uploads =.*/max_file_uploads = 100/" /etc/php/7.2/fpm/php.ini sed -i "s/;date.timezone.*/date.timezone = Europe\/\Berlin/" /etc/php/7.2/fpm/php.ini sed -i "s/;session.cookie_secure.*/session.cookie_secure = True/" /etc/php/7.2/fpm/php.ini sed -i "s/;opcache.enable=.*/opcache.enable=1/" /etc/php/7.2/fpm/php.ini sed -i "s/;opcache.enable_cli=.*/opcache.enable_cli=1/" /etc/php/7.2/fpm/php.ini sed -i "s/;opcache.memory_consumption=.*/opcache.memory_consumption=128/" /etc/php/7.2/fpm/php.ini sed -i "s/;opcache.interned_strings_buffer=.*/opcache.interned_strings_buffer=8/" /etc/php/7.2/fpm/php.ini sed -i "s/;opcache.max_accelerated_files=.*/opcache.max_accelerated_files=10000/" /etc/php/7.2/fpm/php.ini sed -i "s/;opcache.revalidate_freq=.*/opcache.revalidate_freq=1/" /etc/php/7.2/fpm/php.ini sed -i "s/;opcache.save_comments=.*/opcache.save_comments=1/" /etc/php/7.2/fpm/php.ini sed -i "s/;session.save_path =.*/session.save_path = \"N;700;\/usr\/local\/tmp\/sessions\"/" /etc/php/7.2/fpm/php.ini sed -i "s/;emergency_restart_threshold =.*/emergency_restart_threshold = 10/" /etc/php/7.2/fpm/php-fpm.conf sed -i "s/;emergency_restart_interval =.*/emergency_restart_interval = 1m/" /etc/php/7.2/fpm/php-fpm.conf sed -i "s/;process_control_timeout =.*/process_control_timeout = 10s/" /etc/php/7.2/fpm/php-fpm.conf sed -i '$aapc.enabled=1' /etc/php/7.2/fpm/php.ini sed -i '$aapc.file_update_protection=2' /etc/php/7.2/fpm/php.ini sed -i '$aapc.optimization=0' /etc/php/7.2/fpm/php.ini sed -i '$aapc.shm_size=256M' /etc/php/7.2/fpm/php.ini sed -i '$aapc.include_once_override=0' /etc/php/7.2/fpm/php.ini sed -i '$aapc.shm_segments=1' /etc/php/7.2/fpm/php.ini sed -i '$aapc.ttl=7200' /etc/php/7.2/fpm/php.ini sed -i '$aapc.user_ttl=7200' /etc/php/7.2/fpm/php.ini sed -i '$aapc.gc_ttl=3600' /etc/php/7.2/fpm/php.ini sed -i '$aapc.num_files_hint=1024' /etc/php/7.2/fpm/php.ini sed -i '$aapc.enable_cli=0' /etc/php/7.2/fpm/php.ini sed -i '$aapc.max_file_size=5M' /etc/php/7.2/fpm/php.ini sed -i '$aapc.cache_by_default=1' /etc/php/7.2/fpm/php.ini sed -i '$aapc.use_request_time=1' /etc/php/7.2/fpm/php.ini sed -i '$aapc.slam_defense=0' /etc/php/7.2/fpm/php.ini sed -i '$aapc.mmap_file_mask=/usr/local/tmp/apc/apc.XXXXXX' /etc/php/7.2/fpm/php.ini sed -i '$aapc.stat_ctime=0' /etc/php/7.2/fpm/php.ini sed -i '$aapc.canonicalize=1' /etc/php/7.2/fpm/php.ini sed -i '$aapc.write_lock=1' /etc/php/7.2/fpm/php.ini sed -i '$aapc.report_autofilter=0' /etc/php/7.2/fpm/php.ini sed -i '$aapc.rfc1867=0' /etc/php/7.2/fpm/php.ini sed -i '$aapc.rfc1867_prefix =upload_' /etc/php/7.2/fpm/php.ini sed -i '$aapc.rfc1867_name=APC_UPLOAD_PROGRESS' /etc/php/7.2/fpm/php.ini sed -i '$aapc.rfc1867_freq=0' /etc/php/7.2/fpm/php.ini sed -i '$aapc.rfc1867_ttl=3600' /etc/php/7.2/fpm/php.ini sed -i '$aapc.lazy_classes=0' /etc/php/7.2/fpm/php.ini sed -i '$aapc.lazy_functions=0' /etc/php/7.2/fpm/php.ini sed -i "s/09,39.*/# &/" /etc/cron.d/php (crontab -l ; echo "09,39 * * * * /usr/lib/php/sessionclean 2>&1") | crontab -u root -

Configuring: ramdisk

In order to speed things up, some file locations can be located in RAM. There is a association to the owner, www-data. Find out what the UID od www-data is, and use that. In my system, www-data is a default users with id 33. If yours differs from 33, change it before using this code:
id www-data # results in 33 - if not, change following code accordingly: sed -i '$atmpfs /tmp tmpfs defaults,noatime,nosuid,nodev,noexec,mode=1777 0 0' /etc/fstab sed -i '$atmpfs /var/tmp tmpfs defaults,noatime,nosuid,nodev,noexec,mode=1777 0 0' /etc/fstab sed -i '$atmpfs /usr/local/tmp/apc tmpfs defaults,uid=33,size=300M,noatime,nosuid,nodev,noexec,mode=1777 0 0' /etc/fstab sed -i '$atmpfs /usr/local/tmp/cache tmpfs defaults,uid=33,size=300M,noatime,nosuid,nodev,noexec,mode=1777 0 0' /etc/fstab sed -i '$atmpfs /usr/local/tmp/sessions tmpfs defaults,uid=33,size=300M,noatime,nosuid,nodev,noexec,mode=1777 0 0' /etc/fstab mount -a service php7.2-fpm restart && service nginx restart

Installing and configuring: database

NextCloud uses a database. I choose to install MariaDb, the OSS MySQL version:
apt update && apt install mariadb-server -y mysql_secure_installation
When asked for the root password, press enter. Skip (answer N) the first question about changing the root password; answer Y to all other questions.
mv /etc/mysql/my.cnf /etc/mysql/my.cnf.bak vi /etc/mysql/my.cnf # add to my.cnf (it is now a new file): [server] skip-name-resolve innodb_buffer_pool_size = 128M innodb_buffer_pool_instances = 1 innodb_flush_log_at_trx_commit = 2 innodb_log_buffer_size = 32M innodb_max_dirty_pages_pct = 90 long_query_time = 1 max_heap_table_size= 64M query_cache_type = 1 query_cache_limit = 2M query_cache_min_res_unit = 2k query_cache_size = 64M tmp_table_size= 64M slow-query-log = 1 slow-query-log-file = /var/log/mysql/slow.log [client-server] !includedir /etc/mysql/conf.d/ !includedir /etc/mysql/mariadb.conf.d/ [client] default-character-set = utf8mb4 [mysqld] character-set-server = utf8mb4 collation-server = utf8mb4_general_ci binlog_format = MIXED innodb_large_prefix=on innodb_file_format=barracuda innodb_file_per_table=1
Save the file, and start db creation:
service mysql restart mysql -uroot -p CREATE DATABASE nextcloud CHARACTER SET utf8mb4 COLLATE utf8mb4_general_ci; CREATE USER nextcloud@localhost identified by '[your password]'; GRANT ALL PRIVILEGES on nextcloud.* to nextcloud@localhost; FLUSH privileges; quit;

Configuring: prepare nginx for NextCloud, and to use https

service nginx stop mv /etc/nginx/conf.d/default.conf /etc/nginx/conf.d/default.conf.bak vi /etc/nginx/conf.d/nextcloud.conf fastcgi_cache_path /usr/local/tmp/cache levels=1:2 keys_zone=NEXTCLOUD:100m inactive=60m; map $request_uri $skip_cache { default 1; ~*/thumbnail.php 0; ~*/apps/galleryplus/ 0; ~*/apps/gallery/ 0; } server { server_name YOUR.SERVERNAME.HERE; listen 192.168.4.220:80 default_server; listen [::]:80 default_server; location ^~ /.well-known/acme-challenge { proxy_pass http://127.0.0.1:81; proxy_set_header Host $host; } location / { return 301 https://$host$request_uri; } } server { server_name YOUR.SERVERNAME.HERE; listen 192.168.4.220:443 ssl http2 default_server; listen [::]:443 ssl http2 default_server; root /var/www/nextcloud/; access_log /var/log/nginx/nextcloud.access.log main; error_log /var/log/nginx/nextcloud.error.log warn; location = /robots.txt { allow all; log_not_found off; access_log off; } location = /.well-known/carddav { return 301 $scheme://$host/remote.php/dav; } location = /.well-known/caldav { return 301 $scheme://$host/remote.php/dav; } client_max_body_size 2048M; location / { rewrite ^ /index.php$uri; } location ~ ^/(?:build|tests|config|lib|3rdparty|templates|data)/ { deny all; } location ~ ^/(?:\.|autotest|occ|issue|indie|db_|console) { deny all; } location ~ \.(?:flv|mp4|mov|m4a)$ { mp4; mp4_buffer_size 100m; mp4_max_buffer_size 1024m; fastcgi_split_path_info ^(.+\.php)(/.*)$; include fastcgi_params; include php_optimization.conf; fastcgi_pass php-handler; fastcgi_param HTTPS on; } location ~ ^/(?:index|remote|public|cron|core/ajax/update|status|ocs/v[12]|updater/.+|ocs-provider/.+)\.php(?:$|/) { fastcgi_split_path_info ^(.+\.php)(/.*)$; include fastcgi_params; include php_optimization.conf; fastcgi_pass php-handler; fastcgi_param HTTPS on; fastcgi_cache_bypass $skip_cache; fastcgi_no_cache $skip_cache; fastcgi_cache NEXTCLOUD; } location ~ ^/(?:updater|ocs-provider)(?:$|/) { try_files $uri/ =404; index index.php; } location ~ \.(?:css|js|woff|svg|gif|png|html|ttf|ico|jpg|jpeg)$ { try_files $uri /index.php$uri$is_args$args; access_log off; expires 30d; } } vi /etc/nginx/conf.d/letsencrypt.conf i server { server_name 127.0.0.1; listen 127.0.0.1:81 default_server; charset utf-8; access_log /var/log/nginx/le.access.log main; error_log /var/log/nginx/le.error.log warn; location ^~ /.well-known/acme-challenge { default_type text/plain; root /var/www/letsencrypt; } } vi /etc/nginx/ssl.conf #ssl_certificate /etc/letsencrypt/live/[YOUR.DOMAINNAME]/fullchain.pem; #ssl_certificate_key /etc/letsencrypt/live/[YOUR.DOMAINNAME]/privkey.pem; #ssl_trusted_certificate /etc/letsencrypt/live/[YOUR.DOMAINNAME]/fullchain.pem; ssl_dhparam /etc/ssl/certs/dhparam.pem; ssl_session_timeout 1d; ssl_session_cache shared:SSL:50m; ssl_session_tickets off; ssl_protocols TLSv1.2; ssl_ciphers ECDHE-ECDSA-CHACHA20-POLY1305:ECDHE-RSA-CHACHA20-POLY1305:ECDHE-ECDSA-AES128-GCM-SHA256:ECDHE-RSA-AES128-GCM-SHA256:ECDHE-ECDSA-AES256-GCM-SHA384:ECDHE-RSA-AES256-GCM-SHA384:DHE-RSA-AES128-GCM-SHA256:DHE-RSA-AES256-GCM-SHA384:ECDHE-ECDSA-AES128-SHA256:ECDHE-RSA-AES128-SHA256:ECDHE-ECDSA-AES128-SHA:ECDHE-RSA-AES256-SHA384:ECDHE-RSA-AES128-SHA:ECDHE-ECDSA-AES256-SHA384:ECDHE-ECDSA-AES256-SHA:ECDHE-RSA-AES256-SHA:DHE-RSA-AES128-SHA256:DHE-RSA-AES128-SHA:DHE-RSA-AES256-SHA256:DHE-RSA-AES256-SHA:ECDHE-ECDSA-DES-CBC3-SHA:ECDHE-RSA-DES-CBC3-SHA:EDH-RSA-DES-CBC3-SHA:!DSS; # ssl_ecdh_curve secp384r1; # Android still cannot cope with above, so: ssl_ecdh_curve prime256v1; ssl_stapling on; ssl_stapling_verify on; vi /etc/nginx/header.conf add_header Strict-Transport-Security "max-age=15768000; includeSubDomains; preload;"; add_header X-Robots-Tag none; add_header X-Download-Options noopen; add_header X-Permitted-Cross-Domain-Policies none; add_header X-Content-Type-Options "nosniff" always; add_header X-XSS-Protection "1; mode=block" always; add_header Referrer-Policy "same-origin" always; vi /etc/nginx/proxy.conf proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-Host $host; proxy_set_header X-Forwarded-Protocol $scheme; proxy_set_header X-Forwarded-For $remote_addr; proxy_set_header X-Forwarded-Port $server_port; proxy_set_header X-Forwarded-Server $host; proxy_connect_timeout 3600; proxy_send_timeout 3600; proxy_read_timeout 3600; proxy_redirect off; vi /etc/nginx/optimization.conf fastcgi_read_timeout 3600; fastcgi_buffers 64 64K; fastcgi_buffer_size 256k; fastcgi_busy_buffers_size 3840K; fastcgi_cache_key $http_cookie$request_method$host$request_uri; fastcgi_cache_use_stale error timeout invalid_header http_500; fastcgi_ignore_headers Cache-Control Expires Set-Cookie; gzip on; gzip_vary on; gzip_comp_level 4; gzip_min_length 256; gzip_proxied expired no-cache no-store private no_last_modified no_etag auth; gzip_types application/atom+xml application/javascript application/json application/ld+json application/manifest+json application/rss+xml application/vnd.geo+json application/vnd.ms-fontobject application/x-font-ttf application/x-web-app-manifest+json application/xhtml+xml application/xml font/opentype image/bmp image/svg+xml image/x-icon text/cache-manifest text/css text/plain text/vcard text/vnd.rim.location.xloc text/vtt text/x-component text/x-cross-domain-policy; gzip_disable "MSIE [1-6]\."; vi /etc/nginx/php_optimization.conf fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name; fastcgi_param PATH_INFO $fastcgi_path_info; fastcgi_param modHeadersAvailable true; fastcgi_param front_controller_active true; fastcgi_intercept_errors on; fastcgi_request_buffering off; fastcgi_cache_valid 404 1m; fastcgi_cache_valid any 1h; fastcgi_cache_methods GET HEAD;
Now you must remove the leading ‘#’ in the /etc/nginx/nginx.conf file in order to allow to load all files just created. And actually load them, of course.
service nginx restart

Install: NextCloud

Finally.... Get the tarball, unzip it, and surf to the location to do the configuration:
cd /usr/local/src wget https://download.nextcloud.com/server/releases/latest.tar.bz2 tar -xjf latest.tar.bz2 -C /var/www rm latest.tar.bz2 chown -R www-data:www-data /var/www/
Fire up the graphical installer at localhost, choose a name and password for the administrator account, fill in the database credentials and file locations, and you're done. There's an example here.

Install: Certificates

Get free certificates from Letsencrypt:
add-apt-repository ppa:certbot/certbot -y apt update && apt install letsencrypt -y service nginx stop letsencrypt certonly --standalone --agree-tos --rsa-key-size 4096 -d [YOUR.DOMAIN] -d www.[YOUR.DOMAIN] -d cloud.[YOUR.DOMAIN]
If you succeed in getting the certificates, uncomment the first three lines of /etc/nginx/ssl.conf - where the location of the certificates is mentioned, now that you have them.

Some changes in permissions (you might want to create a shell script for this...)
find /var/www/ -type f -print0 | xargs -0 chmod 0640 find /var/www/ -type d -print0 | xargs -0 chmod 0750 chown -R www-data:www-data /var/www/ chown -R www-data:www-data /upload_tmp/ chown -R www-data:www-data /var/nc/ chmod 0644 /var/www/nextcloud/.htaccess chmod 0644 /var/www/nextcloud/.user.ini chmod 600 /etc/letsencrypt/live/[YOUR.DOMAIN]/fullchain.pem chmod 600 /etc/letsencrypt/live/[YOUR.DOMAIN]/privkey.pem chmod 600 /etc/letsencrypt/live/[YOUR.DOMAIN]/chain.pem chmod 600 /etc/letsencrypt/live/[YOUR.DOMAIN]/cert.pem chmod 600 /etc/ssl/certs/dhparam.pem shutdown -r now
Now you should be using https (and getting redirected, if you do not type "https://" in the browser).

Install option: Redis cache server

Simple, but effective:
apt update && apt install redis-server php-redis -y cp /etc/redis/redis.conf /etc/redis/redis.conf.bak
Alter/check/add /etc/redis/redis.conf 1) from: port 6379 to: port 0 2) from: #unixsocket /var/run/redis/redis.sock #unixsocketperm 700 to: unixsocket /var/run/redis/redis.sock unixsocketperm 770 3) from: # maxclients 10000 to: maxclients 512 vi /etc/sysctl.conf add at bottom: vm.overcommit_memory = 1 vi /etc/rc.local add: sysctl -w net.core.somaxconn=65535 You can monitor the working of redis by the following command:
redis-cli -s /var/run/redis/redis.sock monitor

3 comments:

Sasha Gomanuke said...

What's about Office solution? I mean the greatest GoogleDocs.
As far as i know nobody from OpenSorced guys realized it.

Frank said...

NextCloud, and NextCloud community offer several office applications. Check out https://apps.nextcloud.com/categories/office.
I am, however, not here to promote NextCloud as the silver bullet. It works for me, in combination with my own email server.

I have the feeling I have some of my privacy back, and that was the purpose of this excercise - YMMV

Sasha Gomanuke said...

The matter is that i'm looking for solution too, but Google free products are too good...