GoFAST: Prerequisites and Server Installation

Client Item

Prerequisites

See http://gofast-docs.readthedocs.io/fr/latest/docs-gofast-users/doc-gofast-guide-utilisateurs.html#pre-requis-necessaires-pour-lutilisation-de-gofast

Optional components and configurations

Prerequisites and Server Installation

Architecture

GoFAST 4

OS

CentOS 7 64bits

CMS

Drupal 7

Web Server & Reverse Proxy

Apache 2.4 (php-fpm)

Database

MySQL 8.x

Directorie

OpenLDAP 2.4

DMS

Alfresco 7.4 Community

Research

Apache Solr 8

API Server

Apache Tomcat 9

Workflow (GoFAST 3.6+)

Bonitasoft Community 2021

../_images/Tag-Enterprise.png

GoFAST Comm-Server

Collaborative edition

OnlyOffice 7

Instant messaging (>GoFAST 3. 8)

Matrix-Synapse v1.x

Meeting

Jitsi Meet web 1.0.7322

Reverse Proxy

nginx 1.20

Prerequisites “server”

The GoFAST platform is an enterprise application that requires a high-performance architecture in order to deliver a good load ramp and response times.

Caution

GoFAST is particularly sensitive to I/O performance and RAM sizing.

Caution

Due to I/O sensitivity, special attention must be paid to remote storage. The latter should prefer the SAN type. Avoid any remote WAN and NFS storage.

Note

GoFAST v3 requires 2 VMs by default.

Prerequisites Virtual Machine

The GoFAST platform is intended to run on the following 64-bit Hypervisors (which have not reached their EOL):

  • VMWare ESX 7

  • HyperV

  • KVM

  • HVM (Amazon Web Services)

Note

Imaging into HyperV (or Azure) is currently not possible. A script installation is performed

Caution

  • The GoFAST VM must have a dedicated datastore (LUN).

  • Under VMWARE, choose “RedHat Enterprise” and “paravirtualized SCSI adapter” starting with CentOS 7.4 / GoFAST v3.1.1

VM 2 (GF-COMM) is only included in the Enterprise offer

VM 1 (GF)

VM 2 (GF-COMM)*

Hearts

6+

4+

RAM

16GB+

6GB

Storage

200GB+

50GB

IOPS(FIO)

800+

Network

100Mpbs

100Mpbs

Note

A webconference with 6 users (5 with video) consumes 12Mbps (outgoing), 3Mbps (incoming), 50% CPU

Assessment of storage needs

  • 15 GB of disk space for application and system (partition /)

  • Data Storage Space (partition /var)

Normal use

  • 2 to 3 GB of disk space per ‘beginner’ users for their personal content

  • 4 to 6 GB per Groups and Organisations planned

Intensive use

  • 5 to 10 GB of disk space per ‘beginner’ users for their personal content

  • 10 to 20 GB per Groups and Organisations planned

Disk space simulation for 100 users (x5GB), 10 organizations (x10GB), 15 groups (x10GB) for a total of 765GB

Optional Partitioning

Most of the data is stored in /var. A daily backup is performed in /var/backup`. If the image used does not have sufficient partitioning, we recommend that you do it.

Note

It is highly recommended but not mandatory to create a partition for /var

Caution

If the /var partition is on remote storage (NAS/NFS …) this should not lead to high performance degradation (IOPS) compared to local storage. Any WAN storage should be avoided

Caution

It’s recommended but not mandatory to create a partition for /var/backup` for backup of application data and this mount point should be moved

Caution

Swap partition must be at least 50% of the machine’s physical memory

Networks and Security

Certificate

The GoFAST platform only works with a “server” certificate issued by a certificate authority. The certificate must be in the format .pem, or .crt and .key

Caution

The certificate must be “wildcard”

Note

If GoFAST is only available on the Intranet, the certificate can be generated by Active Directory Certificate Services.

Network Architecture

The GoFAST platform can be installed :

  • preferably on a DMZ so that the platform can be accessed from the outside (extranet, mobile use, …)

  • or on the company’s network (LAN)

  • or on another network accessible via the Internet (pay attention in this case to performance, throughput, routing, …)

Caution

If the organization uses a Proxy to access the Internet or on the LAN, special configurations must be made. Experienced problems unrelated to GoFAST and related to proxy configuration issues may occur

List of ports

Here is a list of ports that must be accessible for input and output.

VM1

VM2

Incoming ports

Outgoing Ports

Platform installation, updating, external RSS feeds, web page import, supervision

x

x

443

Access to the platform in HTTPS and WebDav

x

X

443

Sending notification emails or other internal services

x

25, 465, 636..

Remote Administration (SSH)

x

x

Policy to be defined

Zabbix Supervision

x

x

10051

Centralization of logs (Graylog)

x

x

443(UDP)

Meeting

x

10000/UDP

x

443/TCP*

Access to the platform in IMAPS (OPTIONAL)

x

993

Import of external emails (OPTIONAL)

x

25

* if udp ports cannot be opened

Caution

If the subscriber is not on the LAN, the subscriber’s Internet access must pass through the 10000/udp port. This may cause problems on some InternetPro networks, e.g. remote sites connected to the GoFAST server headquarters. Note that this problem is rarely found on 4G networks.

WebRTC Resources (Enterprise only)

Enterprise only

How Webrtc works is described here: https://developer.mozilla.org/fr/docs/Web/Guide/API/WebRTC/WebRTC_architecture

A network diagnostic is possible here: https://www.netscan.co/ (click on “Scan my Network”, the diagnostic is stored as a URL https://www.netscan.co/r/ayiIL )

Note

Unlike video chat, jitsi-meet does not require a STUN server.

Installation

Installation by import of VM

CEO-Vision can provide an image of the VM in OVA format (universal format). For other formats please contact us.

In the case of the OVA format, a virtual machine with standard prerequisites is provided.

Caution

For VMWare, check that the “PVSCSI adapter” is used correctly

In some cases the VM may have more disk space allocated than the VM partitioning. In this case, the following operations must be carried out:

# fdisk -l /dev/sda
Disk /dev/sda: 68.7 GB, 68719476736 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 8354 66999082+ 8e Linux LVM

# fdisk /dev/sda
n                                       {new partition}
p                                       {primary partition}
3                                       {new partition number}
t                                       {change partition id}
8e                                      {Linux LVM partition}
w

reboot

# fdisk -l /dev/sda
Disk /dev/sda: 85.8 GB, 85899345920 bytes
255 heads, 63 sectors/track, 10443 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes
Device Boot Start End Blocks Id System
/dev/sda1 * 1 13 104391 83 Linux
/dev/sda2 14 8354 66999082+ 8e Linux LVM
/dev/sda3 8355 10443 16779892+ 8e Linux LVM

# pvcreate /dev/sda3
# vgdisplay
# vgextend VolGroup00 /dev/sda3 {change VolGroup00 by vgdisplay result
# lvextend /dev/VolGroup00/LogVol00 /dev/sda3 {change accordingly by vgdisplay result

# resize2fs /dev/VolGroup00/LogVol00 {for ext4 and change VolGroup00 accordingly by vgdisplay result}
**OR**
# xfs-growthfs /dev/VolGroup00/LogVol00 {for xfs and change VolGroup00 accordingly by vgdisplay result}

Caution

the last operation can take between 30min and 1h30 for an increase of 1TB

Script installation (Enterprise only)

Enterprise only

Note that this type of installation generates an additional cost (invoicing).

In this case, a VM with Centos 7 x64 Minimal is made available to CEO-Vision who then installs it by script.

Post-installation of the VM (Enterprise only)

Enterprise only

Network configuration (by the Operator)

  • Connect to the VM console and run nmtui

  • Choose Edit on connection and then the interface, normally ems33

  • Click on Show at IP v4 Configuration

  • Enter information (Manual, Addresses, Gateway, DNS servers)

Note

If your GoFAST is accessible from the Internet, the address is a public IP

Note

In a virtualized environment, the gateway is the IP address of the host with the ending replaced by .254

  • Select OK

  • Choose Set system hostname` normally the same name as the one entered into the DNS

  • Exit the application

  • reboot

  • When reconnecting, check that Internet is available ping 8.8.8.8

Note

For ESX, verify that the MAC address matches the configured public IP address

Configuration / Settings (by the Operator)

Note

These steps are very important and should be done as early as possible by the operator/manager

  • Deposit the wildcard SSL certificate on the server (public and private key corresponding to *.mydomain.tld, e.g. gofast.ceo-vision.com)

  • Enter GoFAST’s IP address and domain name into the company’s DNS

  • Create DNS entries :

    • To IP VM1

      • gofast.mydomain.tld (e.g. ``gofast.ceo-vision.com`)

      • gofast-mobile.mydomain.tld (e.g. : gofast-mobile.ceo-vision.com)

    • To VM2 IP

      • gofast-comm.mydomain.tld (e.g. : gofast-comm.ceo-vision.com)

  • Verify that the GoFAST platform has Internet access

  • Verify that all interconnections between the VM work on the ports listed in the “Port List” paragraph

  • Provide remote administration access information (SSH, SMTP gateway …) to CEO-Vision

Configuration / Settings by CEO-Vision

  • Verification of prerequisites

  • Benchmark

  • OPTIONAL: Configuration with the proxy

  • OPTIONAL: Link to OpenLDAP or ActiveDirectory directory (optional)

  • OPTIONAL: Configuring Authentication Delegation

  • SMTP Configuration

  • Installation of the graphics card

  • Configuration of Zabbix supervision probes

  • Installation of APM probes (XXL edition)

Post-installation checks CEO-Vision

Basic performance check

Installation of the tool:

GoFAST v1.x et 2.x
$ sudo yum install http://dl.fedoraproject.org/pub/epel/6/x86_64/epel-release-6-8.noarch.rpm

GoFAST v3.x et +
$ sudo yum install http://dl.fedoraproject.org/pub/epel/7/x86_64/e/epel-release-7-5.noarch.rpm

$ sudo yum install sysbench

Commands to get the software and hardware environment during the bechmark:

# sudo dmidecode | egrep -i 'product name'
# lscpu
# more /etc/centos-release
# df -T
# sudo lshw -class disk -class storage
# lsblk
CPU

Measure the performance of your CPU by doing the following:

Pour sysbench 1.0+:
sysbench cpu --time=0 --events=10000 --threads=4 run

Example of result (in seconds, smaller better):

execution time (avg/stddev): 21.4200/0.00
File IO Benchmark

To measure the performance of the I/O (input/output) it is necessary to create a file much larger than the available RAM because otherwise the system uses the memory as a cache which distorts the results - 150GB is a good value not always usable (lack of disk space):

sysbench --test=fileio --file-total-size=50G prepare

Then run the benchmark:

Pour sysbench 1.0+:
sysbench fileio --file-total-size=50G --file-test-mode=rndrw --time=300 --max-requests=0 --threads=4 run

Then execute the benchmark:

Read 595.16Mb Written 396.77Mb Total transferred 991.92Mb (3.3056Mb/sec)211.56 Requests/sec executed

Then you can delete the test file:

sysbench --test=fileio --file-total-size=50G cleanup
File IO Benchmark (FIO)
yum install fio

fio -filename=/var/TESTIO -iodepth=64 -ioengine=libaio -direct=1 -rw=randrw -bs=4k -size=5G -numjobs=4 -runtime=30 -group_reporting -name=test-randwrite --rwmixread=30

Nb: ajouter --unified_rw_reporting=1 si l'on veut un résultat aggloméré pour Read et Write
MySQL Benchmark

To measure the performance of the MySQL database, we first need to create a table test in the database test (create manually) with 1,000,000 rows of data:

sudo mysql -u root -p -e "CREATE DATABASE test;"

Pour sysbench 1.0+:
sysbench --test="/usr/share/sysbench/tests/include/oltp_legacy/oltp.lua" --db-driver=mysql --oltp-table-size=1000000 --mysql-db=test --mysql-user=root --mysql-password=mypassword prepare

Then run the benchmark:

sysbench --test="/usr/share/sysbench/tests/include/oltp_legacy/oltp.lua" --db-driver=mysql --oltp-table-size=1000000 --mysql-db=test --mysql-user=root --mysql-password=YOURDBPWD --time=90 --oltp-read-only=off --max-requests=0 --threads=4 run

Then execute the benchmark:

transactions:                       28235 (871.01 per sec.)

MySQL Indicators in Normal Operation

gofast.ceo-vision.com ~# mysqltuner -u root

>> MySQLTuner 1.1.1 - Major Hayden <major@mhtx.net>
>> Bug reports, feature requests, and downloads at http://mysqltuner.com/
>> Run with '--help' for additional options and output filtering
Please enter your MySQL administrative login: root
Please enter your MySQL administrative password:

-------- General Statistics --------------------------------------------------
[--] Skipped version check for MySQLTuner script
[OK] Currently running supported MySQL version 5.5.28-log
[OK] Operating on 64-bit architecture

-------- Storage Engine Statistics -------------------------------------------
[--] Status: -Archive -BDB -Federated +InnoDB -ISAM -NDBCluster
[--] Data in InnoDB tables: 84M (Tables: 298)
[--] Data in PERFORMANCE_SCHEMA tables: 0B (Tables: 17)
[!!] Total fragmented tables: 301

-------- Security Recommendations -------------------------------------------

[OK] All database users have passwords assigned

-------- Performance Metrics -------------------------------------------------
[--] Up for: 9m 6s (83K q [153.229 qps], 143 conn, TX: 69M, RX: 21M)
[--] Reads / Writes: 81% / 19%
[--] Total buffers: 544.0M global + 15.5M per thread (150 max threads)
[OK] Maximum possible memory usage: 2.8G (49% of installed RAM)
[OK] Slow queries: 0% (12/83K)
[OK] Highest usage of available connections: 6% (9/150)
[OK] Key buffer size / total MyISAM indexes: 128.0M/1.5M
[OK] Key buffer hit rate: 100.0% (56K cached / 0 reads)
[OK] Query cache efficiency: 74.0% (25K cached / 34K selects)
[OK] Query cache prunes per day: 0
[OK] Sorts requiring temporary tables: 0% (0 temp sorts / 2K sorts)
[!!] Joins performed without indexes: 8
[!!] Temporary tables created on disk: 39% (217 on disk / 550 total)
[OK] Thread cache hit rate: 93% (9 created / 143 connections)
[OK] Table cache hit rate: 98% (455 open / 462 opened)
[OK] Open file limit used: 6% (263/4K)
[OK] Table locks acquired immediately: 100% (27K immediate / 27K locks)
[OK] InnoDB data size / buffer pool: 84.8M/150.0M

-------- Recommendations -----------------------------------------------------

General recommendations:
    Run OPTIMIZE TABLE to defragment tables for better performance
    MySQL started within last 24 hours - recommendations may be inaccurate
    Adjust your join queries to always utilize indexes
    When making adjustments, make tmp_table_size/max_heap_table_size equal
    Reduce your SELECT DISTINCT queries without LIMIT clauses

Variables to adjust:
    join_buffer_size (> 10.0M, or always use indexes with joins)
    tmp_table_size (> 200M)
    max_heap_table_size (> 200M)

APPENDIX I: Common Issues

No sending of technical mails

Configure SMTP relay in sendmail (/etc/postfix/main.cf)

relayhost = smtp.myorganisation.xxx # nom du relais

Unable to connect to GoFAST via email (IMAPS)

The next test must work (end of message ” * OK IMAP4rev1 Server GreenMail ready”) from an internal or external server if the GoFAST is on a DMZ

# openssl s_client -connect gofast.mydomain.tld:993

If not, verify that:

  • port 993 is properly opened (see § on checking open ports)

  • the stunnel process is working