Category: Uncategorized

Install Apache & Set Up Virtual Hosts

This is just a quick step by step guide taken from DigitalOcean's tutorial (see the references at the bottom). The main difference is that it has less explanation - so if you know what you are doing you can just copy & paste the commands ...

$ sudo apt-get update
$ sudo apt-get install apache2

Set Global ServerName to Suppress Syntax Warnings

Next, we will add a single line to the /etc/apache2/apache2.conf file to suppress a warning message. While harmless, if you do not set ServerName globally, you will receive the following warning when checking your Apache configuration for syntax errors:

$ sudo apache2ctl configtes
$ sudo nano /etc/apache2/apache2.conf

Add `ServerName server_domain_or_IP` save and close the file.

Adjusting the Firewall

$ sudo ufw app list
$ sudo ufw allow 'Apache'
$ sudo ufw status

If `ufw` is not enabled do it by running `sudo ufw enable`. Make sure that OpenSSH is on the allowed list.

Checking your Web Server

$ sudo systemctl status apache2

Setting Up Virtual Hosts

Create the directory for your_domain as follows:

$ sudo mkdir /var/www/your_domain
$ nano /var/www/your_domain/index.html

Add to index.html:

<html>
<head>
<title>Welcome to Your_domain!</title>
</head>
<body>
<h1>Success! The your_domain virtual host is working!</h1>
</body>
</html>

In order for Apache to serve this content, it's necessary to create a virtual host file with the correct directives. Instead of modifying the default configuration file located at /etc/apache2/sites-available/000-default.conf directly, let's make a new one at /etc/apache2/sites-available/your_domain.conf:

$ sudo nano /etc/apache2/sites-available/your_domain.conf

Paste the following to apache config:

<VirtualHost *:80>
ServerAdmin webmaster@localhost
ServerName your_domain
ServerAlias www.your_domain
DocumentRoot /var/www/your_domain
ErrorLog ${APACHE_LOG_DIR}/error.log
CustomLog ${APACHE_LOG_DIR}/access.log combined
</VirtualHost>

Let's enable the file with the a2ensite tool and disable the default one:

$ sudo a2ensite your_domain.conf
$ sudo a2dissite 000-default.conf

Check the apache config and reload the apache:

$ sudo apache2ctl configtest
$ sudo systemctl reload apache2

On your local computer edit hosts file:

$ sudo nano /etc/hosts

Add e.g `<server_IP> testtist.xyz` entry, save and quit. Open your web browser and go to `http://testtist.xyz`. You should see `Success! The testtist.xyz virtual host is working!` (i.e. Virtual Hosts is correctly configured).

Ref:
- https://www.digitalocean.com/community/tutorials/how-to-install-linux-apache-mysql-php-lamp-stack-on-ubuntu-16-04
- https://www.digitalocean.com/community/tutorials/how-to-install-the-apache-web-server-on-ubuntu-18-04

Pipenv: basic overview

Pipenv essentially acts as a replacement for pip. It introduces 2 "new" files: Pipfile ("replacement" for requirements.txt) and Pipfile.lock which ensures that builds are deterministic.

Pipenv uses pip and virtualenv under the hood but simplifies their usage with a single command line interface.

First, let’s install it:

$ pip install pipenv

Now let's create a new project:

$ pipenv --python 3.6

This will create a virutalenv for this project and Pipfile file.

Next, we add some dependencies:

$ pipenv install [package]

Once you are done installing packages you need to lock the dependencies:

$ pipenv lock

This will create the Pipfile.lock file where all package names, together with its version and a list of its own dependencies are frozen. You should not manually edit this file.

To uninstall a package simply type:

$ pipenv uninstall [package]

Or to uninstall everything from the env type:

$ pipenv uninstall --all

And to remove the virtualenv type:

$ pipenv --rm

Check for security vulnerabilities (and PEP 508 requirements) in your environment:

$ pipenv check

 

If you want to install packages directly from github:

# install specific tag
$ pipenv install git+https://github.com/karantan/ansible@v2.6.0.1#egg=ansible

# install from latest master
$ pipenv install git+https://github.com/karantan/ansible#egg=ansible

 

 

Further reading:

Conda cheat sheet

Create new environment:

conda create --name sandbox python=3.5

Create the environment from the environment.yml file:

conda env create -f environment.yml

Activate specific environment:

source activate sandbox

Export your active environment to a new file:

conda env export > environment.yml

Remove specific environment:

conda env remove --name sandbox

 

tmux cheatsheet

My personal tmux cheat sheet

Create a new session:

tmux new -s myname

Attach:

tmux a

Attach to a specific named session:

tmux a -t myname

List of sessions:

tmux ls

Tmux uses ctrl + b prefix modifier. After that you can use options like:

d detach sessions
c create window
w list windows
n next window
p previous window
f find window
, name window
& kill window

Scrolling:

  • ctrl + b -> page up/down
  • ctrl + b -> fn + arrow up/down

Python3 strings

A bit of background on unicode and UTF-8:

Unicode has a different way of thinking about characters. In Unicode, the letter “A“ is a platonic ideal. It’s just floating in “heaven”. Every platonic letter in every alphabet is assigned a magic number by the Unicode consortium which is written like this: U+0639 (in python “\u0639“).

UTF-8 is a system of storing your string of unicode code points (those magic “U+number“) in memory using 8 bit bytes.

One of the common questions for python 3 is when to use bytestring and when to use strings as an object? When you are manipulating string (e.g. “reversed(my_string)“) you always use string object and newer bytestring. Why? Here is an example:

 

my_string = "I owe you £100"
my_bytestring = my_string.encode()

>>> print(''.join([c for c in reversed(my_string)]))
001£ uoy ewo I
>>> print(''.join([chr(c) for c in reversed(my_bytestring)]))
001£Â uoy ewo I

You should never call encode without specifying which encoding to use because then the interpreter will pick for you which will “almost” always be UTF-8 but there are some instances where this won’t be so and you will spent a lot of time finding this bug. So ALWAYS specify which encoding to use (e.g. “.encode(‘utf-8’)“). Example:

 

>>> print('I owe you £100'.encode('utf-8').decode('latin-1'))
I owe you £100

Full article can be found here: link

Python PDB

Last month I attended PyMunich conference and in this blog post I want to highlight a talk by Philip Bauer (“Debug like a pro. How to become a better programmer through pdb-driven development“).

Basic commands for pdb that Philip highlighted:

  •  l[ist] (list source code of current file)
  • n[ext] (continue execution until next line)
  • s[tep] (execute the current line, stop at the first possible occasion)
  • r[eturn] (continue execution until the current function returns)
  • c[ontinue] (continue execution, only stop when a breakpoint is encountered)
  • w[here] (show stack trace, recent frame at bottom)
  • u[p] (move up the stack)
  • d[own] (move down the stack)
  • b[reakpoint] (set a new breakpoint. `tbreak` for temporary break points)
  • a[rgs] (print the argument list of the current function)

`ipdb`/`pdbpp` also have long list method (`ll`) which displays the whole function you are in.

Other python debugging tricks you should know about are:

  • use ?for getting additional information lib/class/function/... (e.g. os?)
  • use ??for displaying the source code of the lib/class/function you want to inspect (e.g. os.path.join??)
  • pp(Pretty-print) is already in pdb so you should always use it
  • pp locals()will pretty print local variables

One of the best tricks is the `help` function which accepts object and returns generated help page for the object. !help(obj.__class__)command will generate help page which will contain all the methods including class methods and static methods with docstrings, method resolution order, data descriptors, attributes, data and other attributes inherited and much more.

Full article about PyMunich 2016 is located here: link.

Releasing Python Packages

When you need to fix a bug, add feature or change existing functionality in a library that you are using there are several ways to do this. Here I will show you how to do this properly:

  1. Fork repository.
  2. Clone into your local dir.
  3. Create new branch and add your code (do not forget to add tests!).
  4. Commit & push, create PR on your forked repo, merge.
  5. Create PR to the base project so this change can go upstream (and then the whole community benefits from it).
  6. Release your updated version of this lib.

Step 3 can be a bit tricky to do because often you need to test this new code in your existing project. My favourite way to test if your new code works is to install this lib (package) in "develop mode":

pip install -e /src/my_modified_lib

Step 6 is needed if you have continuous integration like TravisCI. For this you also need your own private pypi server or have an account on official python pypi server. To create a release package that you can upload on pypi server just type:

python setup.py sdist --formats=zip

When releasing package, MANIFEST.in file is very important so if your files are missing in the .zip package, the problem is with MANIFEST.in file.

Then just upload this package (located in `dist` folder) to pypi server. Next thing you will need to do is to tell your buildout process where this new package is located. This is unfortunately very different based on which buildout tool you use (e.g. if you are using buildout tool you need to add link to this package to the "find-links" list).

How to mock ‘open’ built-in function

Here is a code snippet on how to test open built-in function:

Code to test:

def my_function(self, id: int) -> None:
    file_path = 'path/to/files/{}.txt'.format(id)
    with open(file_path, 'w') as f:
        f.write('< My data >')

 

Unit test:

def test_my_function(self):
    _open = mock.mock_open()	
    with mock.patch('my_project.scripts.my_script.open', _open, create=True): # noqa
        my_function(1)
  
    _open.assert_called_with('path/to/files/123.txt', 'w')
    fp = _open.return_value.__enter__.return_value
    fp.write.assert_called_with('< My data >')

 

Happy unit testing.

 

How to mock __import__ built-in function

So lets say we need to write unit test for piece of code that uses  __import__  built-in function:

def task_we_want_to_test(task_name: str, task_class: str, env: dict) -> Task:
    module = __import__(
        'my_project.scripts.{}'.format(task_name),
        fromlist=[task_class, ])
    klass = getattr(module, task_class)
    return klass(env)

 

You can test this code like this:

def test_get_task(self):
    from my_project.scripts import task_we_want_to_test

    MyScript = mock.MagicMock()
    _import = mock.MagicMock()
    _import.MyScript = MyScript
    with mock.patch.dict(
        'sys.modules',
        {'my_project.scripts.my_script': _import},
    ):
        klass = task_we_want_to_test('my_script', self.env)
        MyScript.assert_called_with(self.env)
        self.assertEqual(klass, MyScript(self.env))

 

Git branch naming convention

The naming convention, I propose, have four sections:

  1. Story Type:
    • add == Add Feature
    • fix == Fix, Hotfix, Bug, ...
    • clean == Cleanup, Chore, ...
    • modify  == Modify existing functionality
  2. Short Summary: 2-3 words about what the branch contains
  3. Tracker ID (Planio, GitHub, ... )

First section is group and is separated with dash. 2nd and 3rd sections are separated with hyphens. The end result looks like this:

{story type}/{2-3 word summary}-{tracker id}

 

Navigation