Sync a complete website to remote server quickly by Phing

The famous Phing build tool (written in PHP) offers automation for repetitive tasks, which need to be executed by the Linux terminal. So you can sync an entire website (files and database) to the remote server. Here is an example of a Phing target to do the job:

This target is later contained in your build.xml file. Afterwards you can execute the following command:

phing syncWebsiteToLive

The output will be the following:

Buildfile: /home/peter/build.xml
DevOps > syncWebsiteToLive:
     [echo] Syncing data to live server.
sending incremental file list
db_dumps/latest.sql
      7,515,285 100%  111.94MB/s    0:00:00 (xfr#1, ir-chk=1009/1040)
     [echo] Delete old database
     [echo] Create new empty database
     [echo] Import database dump
BUILD FINISHED
Total time: 5.7706 seconds

The website has been single by one single console command in an absolute minimum of time. This approach is good for a website in development stadium or a website which database can be just replaced. Then this way can save a lot of time compared to other approaches like ftp and manual database dump import.

Sync a complete website to remote server quickly by Phing

The famous Phing build tool (written in PHP) offers automation for repetitive tasks, which need to be executed by the Linux terminal. So you can sync an entire website (files and database) to the remote server. Here is an example of a Phing target to do the job:

This target is later contained in your build.xml file. Afterwards you can execute the following command:

phing syncWebsiteToLive

The output will be the following:

Buildfile: /home/peter/build.xml
DevOps > syncWebsiteToLive:
     [echo] Syncing data to live server.
sending incremental file list
db_dumps/latest.sql
      7,515,285 100%  111.94MB/s    0:00:00 (xfr#1, ir-chk=1009/1040)
     [echo] Delete old database
     [echo] Create new empty database
     [echo] Import database dump
BUILD FINISHED
Total time: 5.7706 seconds

The website has been single by one single console command in an absolute minimum of time. This approach is good for a website in development stadium or a website which database can be just replaced. Then this way can save a lot of time compared to other approaches like ftp and manual database dump import.

Sync a complete website to remote server quickly by Phing

The famous Phing build tool (written in PHP) offers automation for repetitive tasks, which need to be executed by the Linux terminal. So you can sync an entire website (files and database) to the remote server. Here is an example of a Phing target to do the job:

This target is later contained in your build.xml file. Afterwards you can execute the following command:

phing syncWebsiteToLive

The output will be the following:

Buildfile: /home/peter/build.xml
DevOps > syncWebsiteToLive:
     [echo] Syncing data to live server.
sending incremental file list
db_dumps/latest.sql
      7,515,285 100%  111.94MB/s    0:00:00 (xfr#1, ir-chk=1009/1040)
     [echo] Delete old database
     [echo] Create new empty database
     [echo] Import database dump
BUILD FINISHED
Total time: 5.7706 seconds

The website has been single by one single console command in an absolute minimum of time. This approach is good for a website in development stadium or a website which database can be just replaced. Then this way can save a lot of time compared to other approaches like ftp and manual database dump import.

Linux: See specific processes

ps aux | grep drush

Installing free of charge SSL certificate from Letsencrypt.org in Apache 2 webserver on Ubuntu 14.04

SSL certificates are cost-free nowadays. Letsencrypt.org is providing a service which lets you get SSL certificates that are free and work in the common web browsers. The following commands must be executed on the same machine on which your webserver is running. Because letsencrypt.org must be allowed to access it. Download the certbot from letsencrypt.org and make the binary executable:

wget https://dl.eff.org/certbot-auto
chmod a+x certbot-auto

Now temporary stop your Apache server, to download the certificates.

sudo service apache2 stop

Now run the certbot program to download the certificates:

./certbot-auto certonly

This program leads you trough a short form process within your terminal. Asking you for the domain you want to secure.

Mind the SSL certificate renewal

The SSL certificate is valid for 3 months. You will get notified about this via the terminal output from the certbot. It looks like this, if you have executed the "./certbot-auto certonly" command at 23.07.2016:

IMPORTANT NOTES:
 - Congratulations! Your certificate and chain have been saved at
   /etc/letsencrypt/live/my-domain.com/fullchain.pem. Your cert will
   expire on 2016-10-21. To obtain a new or tweaked version of this
   certificate in the future, simply run certbot-auto again. To
   non-interactively renew *all* of your certificates, run
   "certbot-auto renew"
 - If you like Certbot, please consider supporting our work by:
   Donating to ISRG / Let's Encrypt:   https://letsencrypt.org/donate
   Donating to EFF:                    https://eff.org/donate-le

There's an automatic way to accomplish the renewal: https://community.letsencrypt.org/t/how-to-automatically-renew-certificates/4393. Otherwise just make you a calendar entry and run the command from above again.

Enable the SSL certificate on your web server

Enable SSL in your Apache setup:

sudo a2enmod ssl

Now as you have the certificates, you can integrate them in your Apache 2 VHost configuration at /etc/apache2/sites-available/. Change "example.com" to your domain name.

   SSLEngine on
   SSLCertificateFile    /etc/letsencrypt/live/example.com/cert.pem
   SSLCertificateKeyFile /etc/letsencrypt/live/example.com/privkey.pem
   SSLCertificateChainFile /etc/letsencrypt/live/example.com/fullchain.pem 

Switch the virtual host port from 80 to 443. For this set the following in your Vhost configuration file:
to

Small tweak for SEO

Search engines expect that a web page can be accessed by only one address on your website. To redirect all requests from http to https and ensure that only http://your-domain.com will be requested and not http://your-domain.com "and" http://www.your-domain.com, you can setup the redirect within your VHost file. Add this to the top of the configuration file for your virtual host:

Now restart Apache and happy SSL usage:

sudo service apache2 restart

MySQL: Import database dump

Firstly make sure that the "empty" database exists. Then import it:

mysql -uUSERNAME -p"PASSWORD" database_name < dump-filename.sql

Import a Gzip compressed database dump file:

zcat database-dump-file.sql.gz | mysql -uroot -ppassword database-name 

Git: Changing remote repository from one to the other

If you switch your Git repository from one to the other, you need to switch the address from the remote repository. To do so, you need the following command:

git remote set-url origin ssh://user@host:port/folder/repository-name.git

MySQL: dump database and compress it with Gzip

For accomplishing this, execute the following command:

mysqldump -u username -p"password" database_name | gzip > $(date +"%Y-%m-%d")-database_name.sql.gz

This command will generate you a compressed dump file with the date information in the filename.

Drupal 8: Execute automated tests and debug them in PhpStorm

To test Drupal 8 tests by the command line, you need to do the following. You need to export two environment variables first:

export SIMPLETEST_DB=mysql://root:password@localhost/database;
export SIMPLETEST_BASE_URL=http://host;

Then you can execute the test by the following command, being in the Drupal root folder:

vendor/bin/phpunit -c core modules/my_module/tests/src/Kernel/MyTest.php

Make sure to set the environment variable for PhpStorm:

export XDEBUG_CONFIG="remote_enable=1 remote_mode=req remote_port=9000 remote_host=127.0.0.1 remote_connect_back=0"

phpunit.xml 

There's a phpunit.xml.dist file in PHP projects. That's a template for your PHPUnit configuration. You can specifiy there some PHP ini settings. By this method you do not need to export the variables like above. Copy it to phpunit.xml and set there your settings like follows:

Afterwards you are able to run your test without the manual environment variables setting, just by specifying your configuration directory by the "-c" parameter. Example:

vendor/bin/phpunit -c modules/permissions_by_term modules/permissions_by_term/tests/src/Kernel/SelectTermTest.php

This command cannot execute SimpleTest tests, which inherit from the WebTestBase class. Only unit tests and tests which inherits from the KernelTestBase class can be run.

Runs a SimpleTest

// --- SimpleTests
php core/scripts/run-tests.sh --url http://d8-paragraphs:8888 --browser --verbose --color --concurrency 4 --class 'Drupal\paragraphs\Tests\ParagraphsPreviewTest'

Troubleshooting

If the debug mode is switched on in PhpStorm and the debugger does not stop at the breakpoint and you are getting the following output in the debugger window:

Remote file path '/home/user/Websites/my-project/-' is not mapped to any file path in project

If you have already setup path mapping, then there's probably any error in your code. Chances are, that it is in the first line of the setUp() method of your test class, which inherits from KernelTestBase. Then you probably do not have Drupal 8.2.x but Drupal 8.1.x. If it's in the first line, you can insert any code line or insert

xdebug_break(); 

With this function the debugger will be forced to stop on a specific line.

PhpStorm IDE settings (version 2016.1.2)

If you are executing the tests from the command line (and probably also via the IDE itself) it's important that you setup the path mappings, so that Xdebug can recognize your test's location. You can to this in PhpStorm's settings at "Languages & Frameworks > PHP > Servers". Create a host for your Drupal website host and set the path mapping to your tests directory. An example you can find in the screenshot below:

Linux: Get the size of a folder in an easily readable format

The du program on linux shows the sizes from folders nicely. Just type:

du -shc folder_name

 This will show you the folder size in an easy to read output (no bytes, but gigabytes):

peter@computer:/vol$ du -shc folder_name
3.5G    folder_name
3.5G    total

Easily re-format JSON files with the yajl-tools Ubuntu package

If you have a minimized JSON file which you want to reformat, then you can install the yajl-tools package on Ubuntu:

sudo apt-get install yajl-tools

Afterwards you are able to reformat the file with the following command:

json_reformat -s < ugly.json > pretty.json