Jekyll2019-08-14T16:09:00+00:00http://ariera.github.io/feed.xmlAlejandro Riera MainarThis is my personal blog, where I tend to put the new things I have learned. I like it because it helps me avoid the "but I already solved this once" problem. Hopefully you find it useful too :)
The Turrón Experiment2019-06-12T19:05:54+00:002019-06-12T19:05:54+00:00http://ariera.github.io/2019/06/12/the-turron-experiment<p>The turrón experiment is a short project that Eva and I set ourselves to do in our free time, with the only purpose of learning how to properly conduct a scientific experiment, including things like:</p>
<ul>
<li>experimental design
<ul>
<li>specifying our question</li>
<li>putting it in the form of null hypothesis and alternative hypothesis</li>
<li>defining the relevant variables</li>
<li>controlling for confounding factors</li>
<li>choosing in advance the best statistical test and doing a power analysis</li>
</ul>
</li>
<li>data collection</li>
<li>data analysis
<ul>
<li>processing with tools like pandas</li>
<li>plotting with matplotlib and seaborn</li>
<li>statistical tests with statsmodels and sklearn</li>
</ul>
</li>
<li>reporting</li>
</ul>
<p>You can read the full report, the general FAQ, check the data and run the code at:</p>
<h1 style="text-align: center; border:0; margin:0">
<a href="https://github.com/ariera/the-turron-experiment">The Turrón Experiment</a>
</h1>
<h3 style="text-align: center; border:0; margin: 0">
<a href="https://github.com/ariera/the-turron-experiment">https://github.com/ariera/the-turron-experiment</a>
</h3>The turrón experiment is a short project that Eva and I set ourselves to do in our free time, with the only purpose of learning how to properly conduct a scientific experiment, including things like:Data Science Notes: Confidence Intervals2019-06-10T14:18:39+00:002019-06-10T14:18:39+00:00http://ariera.github.io/2019/06/10/data-science-notes-confidence-intervals<p>This year I resolved to start learning Data Science on my free time with the expectation of finding a way to use it in my every day work.</p>
<p>To help me in the process I have been writing some notes in the format of a private blog (which is a recurring <a href="http://datastori.es/120-data-science-with-david-robinson/#t=31:46.163">tip that experts give to begginners</a>). I’ll start publishing my private notes some other day, but today I wanted to write about the lastest item I have read about.</p>
<p><strong>Disclaimer:</strong> The following are more <em>my notes</em> and less <em>a tutorial</em>.</p>
<h2 id="confidence-intervals">Confidence Intervals</h2>
<p>According to <a href="https://stats.stackexchange.com/users/25/harvey-motulsky">Harvey Motulsky</a>, in <a href="https://www.goodreads.com/en/book/show/25923604-essential-biostatistics">Essential Biostatistics</a>:</p>
<p>CIs express precission or margin of errors and so let you <strong>make a general conclusion from limited data</strong>. But this only works under the following assumptions:</p>
<ul>
<li>random sample / representative sample</li>
<li>independent observations</li>
<li>correctly tabulated data / free of bias</li>
</ul>
<h3 id="meaning-of-95-ci">Meaning of 95% CI</h3>
<p>If you calculate the 95% CI of a given observation, you would expect the real population to be encompassed by your CI 95% of the times. But there is no way for you to know whether the real population lies within your CI or not.</p>
<p>We <strong>do not</strong> say <em>there is a 95% chance that the true population is in my CI</em>, that’s flipping things around. The true population is fixed, and there is 95% chance that your CI contains it.</p>
<p>To make this more clear: let’s say you forgot your keys at the kitchen table (but you don’t remember where). If I ask you to guess where your keys are you may say something like <em>“I am 95% sure that I forgot them at home”</em>, which is different from <em>“There is a 95% chance that the keys are at home”</em>. Your keys are 100% at home, whether you know it or not.</p>
<h3 id="a-simulation">A simulation</h3>
<p>Harvey proposes the following exercise to understand better. I added some code to help.</p>
<p>Imagine you have a bowl with 100 balls, 25 of them are red and 75 are black. Pretend you are a researcher who doesn’t know the real distribution.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">red_balls</span> <span class="o">=</span> <span class="p">[</span><span class="s">'red'</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">25</span><span class="p">)]</span>
<span class="n">black_balls</span> <span class="o">=</span> <span class="p">[</span><span class="s">'black'</span> <span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="mi">75</span><span class="p">)]</span>
<span class="n">bowl</span> <span class="o">=</span> <span class="n">red_balls</span> <span class="o">+</span> <span class="n">black_balls</span></code></pre></figure>
<p>Next we mix the balls, choose one randomly and put it back again in the bowl. We repeat this process 15 times and calculate the 95% CI for this proportion:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="kn">import</span> <span class="nn">random</span>
<span class="kn">from</span> <span class="nn">statsmodels.stats.proportion</span> <span class="kn">import</span> <span class="n">proportion_confint</span>
<span class="k">def</span> <span class="nf">simulation</span><span class="p">():</span>
<span class="n">number_of_red_balls_observed</span> <span class="o">=</span> <span class="mi">0</span>
<span class="n">NUMBER_OF_TRIALS</span> <span class="o">=</span> <span class="mi">15</span>
<span class="n">CONFIDENCE_LEVEL</span> <span class="o">=</span> <span class="mf">0.95</span>
<span class="k">for</span> <span class="n">i</span> <span class="ow">in</span> <span class="nb">range</span><span class="p">(</span><span class="n">NUMBER_OF_TRIALS</span><span class="p">):</span>
<span class="n">random</span><span class="o">.</span><span class="n">shuffle</span><span class="p">(</span><span class="n">bowl</span><span class="p">)</span>
<span class="n">ball</span> <span class="o">=</span> <span class="n">random</span><span class="o">.</span><span class="n">choice</span><span class="p">(</span><span class="n">bowl</span><span class="p">)</span>
<span class="k">if</span> <span class="n">ball</span> <span class="o">==</span> <span class="s">'red'</span><span class="p">:</span>
<span class="n">number_of_red_balls_observed</span> <span class="o">+=</span> <span class="mi">1</span>
<span class="n">ci_low</span><span class="p">,</span> <span class="n">ci_up</span> <span class="o">=</span> <span class="n">proportion_confint</span><span class="p">(</span><span class="n">number_of_red_balls_observed</span><span class="p">,</span> <span class="n">NUMBER_OF_TRIALS</span><span class="p">,</span> <span class="n">alpha</span><span class="o">=</span><span class="mf">1.0</span> <span class="o">-</span> <span class="n">CONFIDENCE_LEVEL</span><span class="p">)</span>
<span class="n">observed_proportion</span> <span class="o">=</span> <span class="n">number_of_red_balls_observed</span><span class="o">/</span><span class="n">NUMBER_OF_TRIALS</span>
<span class="k">return</span> <span class="p">(</span><span class="n">ci_low</span><span class="p">,</span> <span class="n">ci_up</span><span class="p">,</span> <span class="n">observed_proportion</span><span class="p">)</span></code></pre></figure>
<p>If we repeat this exercise 20 times we should see that:</p>
<ul>
<li>about half of the times the observed proportion is above the real population</li>
<li>the other half of the times it would be lower</li>
<li>5% of the times the calculate CI will not encompass the real population</li>
</ul>
<p>The following figure shows the 20 confidence intervals in the form of bars, with a line in the middle indicating the observed proportion. A horizontal line show the true proportion (25% of the balls are red).</p>
<p style="text-align: center;">
<img src="http://ariera.github.io/assets/data-sci-notes/confidence_intervals.png" alt="Figure 1: Confidence intervals of 20 samples from a binomial distribution B(15,0.25)" />
<small>Figure 1: Confidence intervals of 20 samples from a binomial distribution B(15,0.25)</small>
</p>
<h4 id="the-code">The code</h4>
<p>You can find and run the full code at: <a href="https://github.com/ariera/essential-biostatistics">github.com/ariera/essential-biostatistics</a></p>
<p><br />
<br /></p>
<div class="hr"></div>
<p><br />
<br />
<script src="https://utteranc.es/client.js" repo="ariera/ariera.github.io" issue-term="pathname" label="comments" theme="github-light" crossorigin="anonymous" async="">
</script></p>This year I resolved to start learning Data Science on my free time with the expectation of finding a way to use it in my every day work.TIL: ssh forwarding with tmux2019-06-03T15:10:22+00:002019-06-03T15:10:22+00:00http://ariera.github.io/2019/06/03/til-ssh-forwarding-with-tmux<p>Here is the simple recipie that I have to search for time and again. Check the sources linked below for a much better explanation of how and why.</p>
<h3 id="in-my-computer">In my computer</h3>
<p>Make your key available to <code class="highlighter-rouge">ssh-agent</code></p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># for mac</span>
ssh-add <span class="nt">-K</span> ~/.ssh/id_rsa</code></pre></figure>
<p>and add the following to <code class="highlighter-rouge">~/.ssh/config</code></p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"> <span class="c"># ~/.ssh/config</span>
Host <span class="k">*</span>
SendEnv LANG LC_<span class="k">*</span>
ForwardAgent Yes</code></pre></figure>
<h3 id="configure-tmux-in-the-remote-server">Configure <code class="highlighter-rouge">tmux</code> in the remote server:</h3>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"> <span class="c"># ~/.tmux.conf</span>
set-environment <span class="nt">-g</span> <span class="s1">'SSH_AUTH_SOCK'</span> ~/.ssh/ssh_auth_sock</code></pre></figure>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"> <span class="c"># ~/.ssh/rc</span>
<span class="k">if</span> <span class="o">[</span> <span class="nt">-S</span> <span class="s2">"</span><span class="nv">$SSH_AUTH_SOCK</span><span class="s2">"</span> <span class="o">]</span><span class="p">;</span> <span class="k">then
</span>ln <span class="nt">-sf</span> <span class="nv">$SSH_AUTH_SOCK</span> ~/.ssh/ssh_auth_sock
<span class="k">fi</span></code></pre></figure>
<h3 id="debbuging-is-ssh-forwarding-working">Debbuging: is ssh forwarding working?</h3>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"> <span class="nb">echo</span> <span class="s2">"</span><span class="nv">$SSH_AUTH_SOCK</span><span class="s2">"</span>
<span class="c"># Should print something like this:</span>
<span class="c"># /tmp/ssh-4hNGMk8AZX/agent.79453</span></code></pre></figure>
<h3 id="sources">Sources</h3>
<ul>
<li><a href="https://werat.github.io/2017/02/04/tmux-ssh-agent-forwarding.html">Happy ssh agent forwarding for tmux/screen @ werat.github.io</a></li>
<li><a href="https://developer.github.com/v3/guides/using-ssh-agent-forwarding/">Using SSH agent forwarding @ developer.github.com</a></li>
</ul>Here is the simple recipie that I have to search for time and again. Check the sources linked below for a much better explanation of how and why.TIL: move files from a git repo to another preserving history2018-10-26T06:22:39+00:002018-10-26T06:22:39+00:00http://ariera.github.io/2018/10/26/til-how-to-move-files-from-a-git-repo-to-another-preserving-history<p>Today I learned how to move files and folders from one git repository to another one while preserving history, thanks to <a href="http://gbayer.com/development/moving-files-from-one-git-repository-to-another-preserving-history/">Greg Bayer</a></p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="c"># First we clone the original repo to a temp location</span>
git clone REPO_A_URL repo-A-temp-dir
<span class="nb">cd </span>repo-A-temp
git remote rm origin
<span class="c"># Then we filter our subdirecotory</span>
git filter-branch <span class="nt">--subdirectory-filter</span> PATH/TO/YOUR/DIR <span class="nt">--</span> <span class="nt">--all</span>
git add <span class="nb">.</span>
git commit
<span class="c"># Now we clone (or create) the destination repository</span>
<span class="nb">cd</span> ..
git clone REPO_B_URL repo-B-dir
<span class="nb">cd </span>repo-B-dir
<span class="c"># Add the filtered subdirectory as a remote a pull from it</span>
git remote add repo-A-branch ../repo-A-temp-dir
git pull repo-A-branch master <span class="nt">--allow-unrelated-histories</span>
git remote rm repo-A-branch</code></pre></figure>
<p>Done :)</p>Today I learned how to move files and folders from one git repository to another one while preserving history, thanks to Greg BayerTIL: How to debug cron jobs2018-02-27T11:05:28+00:002018-02-27T11:05:28+00:00http://ariera.github.io/2018/02/27/til-how-to-debug-cron-jobs<p>Today I learned how to properly debug cron jobs, thanks to <a href="https://serverfault.com/a/85906">https://serverfault.com/a/85906</a>.</p>
<p>They key is to recreate the correct environment in which your scripts get executed. I’m going to do this for <code class="highlighter-rouge">root</code> but you can do this with every user.</p>
<p>First: edit your crontab</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">crontab <span class="nt">-e</span>
<span class="c">### Add this line:</span>
<span class="k">*</span> <span class="k">*</span> <span class="k">*</span> <span class="k">*</span> <span class="k">*</span> /usr/bin/env <span class="o">></span> /root/cron-env</code></pre></figure>
<p>Once it has had the chance to execute once you can remove it.</p>
<p>Second: let’s create a new script called <code class="highlighter-rouge">run-as-cron</code> that will run your scripts with the correct environment cron is running.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">cd</span> /root
touch run-as-cron
chmod +x run-as-cron
vim run-as-cron
<span class="c">### Add these lines:</span>
<span class="c">#!/bin/bash</span>
/usr/bin/env <span class="nt">-i</span> <span class="k">$(</span><span class="nb">cat</span> /root/cron-env<span class="k">)</span> <span class="s2">"</span><span class="nv">$@</span><span class="s2">"</span></code></pre></figure>
<p>Finally, debug your problematic script like this:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">/root/run-as-cron /the/problematic/script <span class="nt">--with</span> arguments <span class="nt">--and</span> parameters</code></pre></figure>Today I learned how to properly debug cron jobs, thanks to https://serverfault.com/a/85906.Django + webpack + Vue.js - setting up a new project that’s easy to develop and deploy (part 1)2017-09-26T06:55:25+00:002017-09-26T06:55:25+00:00http://ariera.github.io/2017/09/26/django-webpack-vue-js-setting-up-a-new-project-that-s-easy-to-develop-and-deploy-part-1<p><strong><span class="warning">UPDATE 2018-11-12:</span> Rodrigo Smaniotto wrote an updated version and more complete than this article using Dajngo2 and Vue CLI3. I can only recommend checking it here: <a href="https://medium.com/@rodrigosmaniotto/integrating-django-and-vuejs-with-vue-cli-3-and-webpack-loader-145c3b98501a">https://medium.com/@rodrigosmaniotto/integrating-django-and-vuejs-with-vue-cli-3-and-webpack-loader-145c3b98501a</a></strong></p>
<hr />
<p><br /></p>
<p><strong><span class="warning">UPDATE:</span> I recently created a working project at <a href="https://github.com/ariera/django-vue-template">ariera/django-vue-template</a> using the latest version of the vue-webpack template and bundling all the changes highlighted in this post in just this commit <a href="https://github.com/ariera/django-vue-template/commit/8a5c552a20bb9ec7e5751adef3b5d2e26addbcf4">8a5c552</a>.</strong> You may also be interested in checking a <a href="https://github.com/CharlesAracil/webpack-vue-django">vue project template that CharlesAracil</a> created taking inspiration from this entry.</p>
<hr />
<p><br /></p>
<p>I recently started a new web project based on both Django and Vuejs. It took a good deal of reading half-related and/or outdated posts, tons of documentation, and a good deal of trial and error to get it right.</p>
<p>In this post we will cover the key points on <strong>setting up a nice and solid development environment that is consisten with production and hence easy to deploy</strong>. The specifics of how to deploy, final configuration touches and (automation) will be discussed in a future article.</p>
<p>This is the solution that I found, there may be better alternatives (which I’d be happy to read) and it is utimately written with the intention of helping others as well as my future-self :)</p>
<h1 class="no-border">
<img src="http://ariera.github.io/assets/django-vue-dev.png" alt="django webpack vue logos" />
</h1>
<h2 class="mt-30">The problem</h2>
<p>Both Django and Vue offer local web servers to facilitate development, but that confused me in the beginning. Does that mean that I need to be running both servers in parallel? And do I need to switch from one to another while developing? That sounds cumbersome! If you are building a full SPA with Vue as your frontend, and Django as a simple API backend that might be okey, but I want something else.</p>
<p>I want to remain flexible: <strong>some parts of my project will behave like an SPA, other parts will be rendered by Django, and some other will be a mixture: Django-rendered with Vue components</strong>. I want to be able to use my webpack-compiled css in my Django templates, as well as other static assets.</p>
<h2 class="mt-30">My goals</h2>
<ol>
<li>minimise differences in between prod & dev</li>
<li>use just one url in development (ie Django’s runserver should proxy to webpack’s dev server, or the other way around)</li>
<li>preserve hot reload (ie. HMR) in development</li>
<li>django backed pages should work without much ceremony</li>
</ol>
<h2 class="mt-30">My solution</h2>
<p>The solution I found is based around the django library <a href="https://github.com/ezhome/django-webpack-loader"><code class="highlighter-rouge">django-webpack-loader</code></a> and the <a href="https://github.com/ezhome/webpack-bundle-tracker"><code class="highlighter-rouge">webpack-bundle-tracker</code></a> webpack plugin. The django module will read a manifest file, and a webpack plugin take care of updating it. That way, webpack will compile all the assets and Django will be able to use them in your templates.</p>
<p>All you will need to do while developing is launch both local web servers and point your browser to the Django one :)</p>
<p><em>Versions i’m using:</em></p>
<ul>
<li>Vue.js 2.4.2</li>
<li>Django 1.11</li>
<li>vue-cli 2.8.2</li>
<li>Python 3.6.1</li>
</ul>
<h1 class="no-border">Setting up the dev environment</h1>
<h2 class="mt-30">Creating the project</h2>
<p>I have used the <a href="https://github.com/vuejs/vue-cli">vue-cli</a> to create a new project based on the <a href="https://vuejs-templates.github.io/webpack/">webpack template</a>, it will do most of the configuration and setup for us. Then, inside the new directory holding my project, I had created a new django project:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">vue init webpack my_project
<span class="nb">cd </span>my_project
django-admin startproject my_project .</code></pre></figure>
<p>Now install both libraries</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">yarn add webpack-bundle-tracker <span class="nt">--dev</span>
pip install django-webpack-loader
pip freeze <span class="o">></span> requirements.txt</code></pre></figure>
<p>Add <code class="highlighter-rouge">django-webpack-loader</code> to your <code class="highlighter-rouge">INSTALLED_APPS</code>, we will get to the specific configuration in a minute.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./my_project/settings.py</span>
<span class="n">INSTALLED_APPS</span> <span class="o">=</span> <span class="p">[</span>
<span class="s">'webpack_loader'</span><span class="p">,</span>
<span class="p">]</span></code></pre></figure>
<p>And configure webpack to use <code class="highlighter-rouge">BundleTracker</code>.</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="c1">// build/webpack.base.conf.js</span>
<span class="kd">let</span> <span class="nx">BundleTracker</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'webpack-bundle-tracker'</span><span class="p">)</span>
<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="na">plugins</span><span class="p">:</span> <span class="p">[</span>
<span class="k">new</span> <span class="nx">BundleTracker</span><span class="p">({</span><span class="na">filename</span><span class="p">:</span> <span class="s1">'./webpack-stats.json'</span><span class="p">}),</span>
<span class="p">],</span>
<span class="p">}</span></code></pre></figure>
<h2 id="defining-the-statics-paths">Defining the statics paths</h2>
<p>This part was one of the trickiest, there are several configuration options all with very similar names and they feel very confusing to new commers to the framework. I’ll spend a little explaining what and why every option does.</p>
<p>My goal here was to have in the end a directory named <code class="highlighter-rouge">./public</code> (at the root of the project) where all the assest will be stored, with no unnecesary subdirectories (ie. grouping all javascript inside a <code class="highlighter-rouge">js</code> dir would be okey, but theres no need for deeper levels). The final <code class="highlighter-rouge">./public</code> folder would look like this</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>- public/
- js/ (webpack generated)
- app.c611f8fd9b27da4ec95f.js
- app.c611f8fd9b27da4ec95f.js.map
- ...
- css/ (webpack generated)
- ...
- img/ (django generated)
- ...
- admin/ (django generated)
- ...
</code></pre></div></div>
<p>In order for this to happen we have to configure both worlds properly. Let’s take a look at the final configuration of both Django and webpack, and then we will discuss them.</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="c1">// ./config/index.js</span>
<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">build</span><span class="p">:</span> <span class="p">{</span>
<span class="na">assetsRoot</span><span class="p">:</span> <span class="nx">path</span><span class="p">.</span><span class="nx">resolve</span><span class="p">(</span><span class="nx">__dirname</span><span class="p">,</span> <span class="s1">'../dist/'</span><span class="p">),</span>
<span class="na">assetsSubDirectory</span><span class="p">:</span> <span class="s1">''</span><span class="p">,</span>
<span class="na">assetsPublicPath</span><span class="p">:</span> <span class="s1">'/static/'</span><span class="p">,</span>
<span class="c1">// ...</span>
<span class="p">},</span>
<span class="na">dev</span><span class="p">:</span> <span class="p">{</span>
<span class="na">assetsPublicPath</span><span class="p">:</span> <span class="s1">'http://localhost:8080/'</span><span class="p">,</span>
<span class="c1">// ...</span>
<span class="p">}</span>
<span class="p">}</span></code></pre></figure>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./my_project/settings.py</span>
<span class="n">STATICFILES_DIRS</span> <span class="o">=</span> <span class="p">(</span>
<span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">BASE_DIR</span><span class="p">,</span> <span class="s">'dist'</span><span class="p">),</span>
<span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">BASE_DIR</span><span class="p">,</span> <span class="s">'static'</span><span class="p">),</span>
<span class="p">)</span>
<span class="n">STATIC_ROOT</span> <span class="o">=</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">BASE_DIR</span><span class="p">,</span> <span class="s">'public'</span><span class="p">)</span>
<span class="n">STATIC_URL</span> <span class="o">=</span> <span class="s">'/static/'</span>
<span class="n">WEBPACK_LOADER</span> <span class="o">=</span> <span class="p">{</span>
<span class="s">'DEFAULT'</span><span class="p">:</span> <span class="p">{</span>
<span class="s">'BUNDLE_DIR_NAME'</span><span class="p">:</span> <span class="s">''</span><span class="p">,</span>
<span class="s">'STATS_FILE'</span><span class="p">:</span> <span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">BASE_DIR</span><span class="p">,</span> <span class="s">'webpack-stats.json'</span><span class="p">),</span>
<span class="p">}</span>
<span class="p">}</span></code></pre></figure>
<p>Let’s analyse this step by step.</p>
<ol>
<li>
<p><strong><code class="highlighter-rouge">build.assetsRoot: path.resolve(__dirname, '../dist/')</code></strong></p>
<p>This is the output directory of our <code class="highlighter-rouge">npm run build</code> task, that takes care of compiling and building all of our webpack-controlled assets. We will run this task as part of the deployment process. It will take care of analysing all our webpack entry points, traversing and compiling them. The out put will be stored in the <code class="highlighter-rouge">./dist</code> directory at the root of our project (which should be gitignored).</p>
</li>
<li>
<p><strong><code class="highlighter-rouge">STATICFILES_DIRS</code></strong> and <strong><code class="highlighter-rouge">STATIC_ROOT</code></strong></p>
<p>When deploying to production we will also run the Django script <code class="highlighter-rouge">collectstatic</code> that takes care of finding all the static files our project makes use of, and putting them in the correct folder. With <code class="highlighter-rouge">STATICFILES_DIRS</code> we are telling it were to look for these files. With <code class="highlighter-rouge">STATIC_ROOT</code> we specify where to place them.</p>
<p>In <code class="highlighter-rouge">STATIC_DIRS</code> we have specified 2 directories: one is the <code class="highlighter-rouge">./static</code> (Django’s default), and the above defined <code class="highlighter-rouge">./dist</code> where webpack will place its compiled assets.</p>
</li>
<li>
<p><strong><code class="highlighter-rouge">build.assetsSubDirectory: ''</code></strong> and <strong><code class="highlighter-rouge">build.assetsPublicPath: '/static/'</code></strong></p>
<p>Now we tell webpack that (in production) any asset reference should point to <code class="highlighter-rouge">/static</code> (ie at the root of the domain). For example, if our code reads:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <img src="images/logo.png">
</code></pre></div> </div>
<p>the final compiled version will be:</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code> <img src="/static/images/logo.png">
</code></pre></div> </div>
</li>
<li>
<p><strong><code class="highlighter-rouge">STATIC_URL = '/static/'</code></strong></p>
<p>The same thing but for Django. Everytime one of our templates referes to a static file it will render it under the <code class="highlighter-rouge">/static</code> path.</p>
</li>
<li>
<p><strong><code class="highlighter-rouge">dev.assetsPublicPath: 'http://localhost:8080/'</code></strong></p>
<p>This is one of the key points in which we achieve our goal number 2 (use only one url in dev). We want the webpack dev server to serve our webpack assets. While we point our browser to Django’s <code class="highlighter-rouge">localhost:8000</code>, webpack will compile our code to point at <code class="highlighter-rouge">localhost:8080</code> (notice those are 2 different ports).</p>
</li>
<li>
<p><strong><code class="highlighter-rouge">WEBPACK_LOADER</code></strong></p>
<p>In <code class="highlighter-rouge">WEBPACK_LOADER['DEFAULT']['STATS_FILE']</code> we point to the json manifest file generated by webpack’s <code class="highlighter-rouge">BundleTracker</code> (see above in file <code class="highlighter-rouge">build/webpack.base.conf.js</code>).</p>
<p>And we set to blank <code class="highlighter-rouge">WEBPACK_LOADER['DEFAULT']['BUNDLE_DIR_NAME']</code> because we don’t want to nest our webpack assets into any subdirectory.</p>
</li>
</ol>
<h3 class="no-border"> Referencing static files from both Vue and Django</h3>
<p>One final touch. I would like to have a place to put images, fonts, or generic statics assets that can be used from both worlds: Django tempaltes and Vue components. In other words, I want to have a <code class="highlighter-rouge">./static</code> directory at the root of the project where I could put (for example) my logo and be able to write this code:</p>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="c"><!-- in a Django template --></span>
<span class="nt"><img</span> <span class="na">src=</span><span class="s">"{% static 'logo.png' %}"</span><span class="nt">></span>
<span class="c"><!-- in a Vue component --></span>
<span class="nt"><img</span> <span class="na">src=</span><span class="s">"static/logo.png"</span><span class="nt">></span></code></pre></figure>
<p>The Django example already works thanks to the config above described (see <code class="highlighter-rouge">STATICFILES_DIRS</code>), but for Vue/Webpack to work you’d need to add an alias to the wbepack config, like this.</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="c1">// build/webpack.base.conf.js</span>
<span class="nx">module</span><span class="p">.</span><span class="nx">exports</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">resolve</span><span class="p">:</span> <span class="p">{</span>
<span class="na">alias</span><span class="p">:</span> <span class="p">{</span>
<span class="c1">// ...</span>
<span class="s1">'__STATIC__'</span><span class="p">:</span> <span class="nx">resolve</span><span class="p">(</span><span class="s1">'static'</span><span class="p">),</span>
<span class="p">},</span>
<span class="p">}</span></code></pre></figure>
<p>Thanks to this we can now write our Vue code like this (mind the <code class="highlighter-rouge">~</code>):</p>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="nt"><img</span> <span class="na">src=</span><span class="s">"~__STATIC__/logo.png"</span><span class="nt">></span></code></pre></figure>
<h2 id="hot-realod-or-hot-module-replacement-hmr">Hot realod or Hot Module Replacement (HMR)</h2>
<p>While in development we are only accessing Django’s server, and proxying our asset requests to the webpack server (see above <code class="highlighter-rouge">dev.assetsPublicPath</code>). Because of this HMR or hot reload is broken. We need to configure webpack’s hot middleware to point to the right path, and we need to add a new header to the webpack dev server so it allows for CORS requests.</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="c1">// build/dev-client.js</span>
<span class="c1">// before</span>
<span class="kd">var</span> <span class="nx">hotClient</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'webpack-hot-middleware/client?noInfo=true&reload=true'</span><span class="p">)</span>
<span class="c1">// after</span>
<span class="kd">var</span> <span class="nx">hotClient</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'webpack-hot-middleware/client?noInfo=true&reload=true&path=http://localhost:8080/__webpack_hmr'</span><span class="p">)</span></code></pre></figure>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="c1">// build/dev-server.js</span>
<span class="kd">var</span> <span class="nx">devMiddleware</span> <span class="o">=</span> <span class="nx">require</span><span class="p">(</span><span class="s1">'webpack-dev-middleware'</span><span class="p">)(</span><span class="nx">compiler</span><span class="p">,</span> <span class="p">{</span>
<span class="na">headers</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">"Access-Control-Allow-Origin"</span><span class="p">:</span><span class="s2">"</span><span class="err">\</span><span class="s2">*"</span>
<span class="p">},</span>
<span class="c1">// ...</span>
<span class="p">})</span></code></pre></figure>
<div class="warning">UPDATE 2017.11.28</div>
<p>The official vue webpack template <a href="https://github.com/vuejs-templates/webpack/commit/aba0f9c7b7903a5371469ecbd80b6f7df93fe536">was updated around 2 weeks ago</a> to start using the official <code class="highlighter-rouge">webpack-dev-server</code>.</p>
<p>In order to properly configure HMR after the update you simply need to add the following to the <code class="highlighter-rouge">build/webpack.dev.conf.js</code>:</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="c1">// build/webpack.dev.conf.js</span>
<span class="nx">devServer</span><span class="p">:</span> <span class="p">{</span>
<span class="nl">headers</span><span class="p">:</span> <span class="p">{</span>
<span class="s2">"Access-Control-Allow-Origin"</span><span class="p">:</span><span class="s2">"</span><span class="err">\</span><span class="s2">*"</span>
<span class="p">},</span>
<span class="c1">// ...</span>
<span class="p">}</span></code></pre></figure>
<p>Thank you very much to <a href="https://forum.vuejs.org/t/dev-client-dev-server/21645">Emilien and LinusBorg for the feedback</a> on this :)</p>
<h2 id="the-basic-django-template">The basic Django template</h2>
<p>We will define a Django template reusable by other parts of our application that will take care of including the webpack assets.</p>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="c"><!-- ./templates/my_project/base.html --></span>
{% load render_bundle from webpack_loader %}
<span class="nt"><html></span>
<span class="nt"><body></span>
{% block content %}
{% endblock %}
{% render_bundle 'app' %}
<span class="nt"></body></span>
<span class="nt"></html></span></code></pre></figure>
<p>Now other parts of our project could just extend this template and benefit from having all the styles from css or javascript accessible. In fact, let’s create a second template, extending this one, to serve as the root of our SPA:</p>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="c"><!-- ./templates/my_project/spa.html --></span>
{% extends "my_project/base.html" %}
{% block content %}
<span class="nt"><div</span> <span class="na">id=</span><span class="s">"app"</span><span class="nt">></div></span>
{% endblock %}</code></pre></figure>
<p>Notice that in order for this template to be detected by Django you need to add the following to your <code class="highlighter-rouge">settings.py</code></p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./my_project/settings.py</span>
<span class="n">TEMPLATES</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span>
<span class="s">'DIRS'</span><span class="p">:</span> <span class="p">[</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">BASE_DIR</span><span class="p">,</span> <span class="s">'templates'</span><span class="p">)],</span>
<span class="c"># ...</span>
<span class="p">}</span>
<span class="p">]</span></code></pre></figure>
<p>And in our <code class="highlighter-rouge">main.js</code> where we define the root component of our SPA, we will point to this <code class="highlighter-rouge">div#app</code>:</p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="k">import</span> <span class="nx">App</span> <span class="k">from</span> <span class="s1">'./App'</span>
<span class="k">new</span> <span class="nx">Vue</span><span class="p">({</span>
<span class="na">el</span><span class="p">:</span> <span class="s1">'#app'</span><span class="p">,</span>
<span class="na">template</span><span class="p">:</span> <span class="s1">'<App/>'</span><span class="p">,</span>
<span class="na">components</span><span class="p">:</span> <span class="p">{</span> <span class="nx">App</span> <span class="p">},</span>
<span class="p">})</span></code></pre></figure>
<p>The last step would be to define a new url route in our Django settings:</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./my_project/urls.py</span>
<span class="kn">from</span> <span class="nn">django.views.generic</span> <span class="kn">import</span> <span class="n">TemplateView</span>
<span class="n">urlpatterns</span> <span class="o">=</span> <span class="p">[</span>
<span class="n">url</span><span class="p">(</span><span class="s">r'^$'</span><span class="p">,</span> <span class="n">TemplateView</span><span class="o">.</span><span class="n">as_view</span><span class="p">(</span><span class="n">template_name</span><span class="o">=</span><span class="s">'my_project/spa.html'</span><span class="p">),</span> <span class="n">name</span><span class="o">=</span><span class="s">'home'</span><span class="p">),</span>
<span class="c"># ...</span>
<span class="p">]</span></code></pre></figure>
<p>And we can now open <code class="highlighter-rouge">http://localhost:8000</code> in our browser and enjoy our fully functional dev environment :)</p>
<h2 id="clean-unnecessary-files-and-options">Clean unnecessary files and options</h2>
<p><strong><span class="warning">UPDATE:</span> It is better ignore everything related to <code class="highlighter-rouge">HtmlWebpackPlugin</code> in this part, it was breakign the end-to-end tests.</strong></p>
<p><del>Before we conclude let’s do a little cleanup.</del></p>
<p><del>Remove the <code class="highlighter-rouge">index.html</code> file at the root of your project and stop using <code class="highlighter-rouge">HtmlWebpackPlugin</code> in development. Since Django will render the initial html and include any assets there we don’t need <code class="highlighter-rouge">HtmlWebpackPlugin</code> to generate an initial html file for us.</del></p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="c1">// build/webpack.dev.conf.js</span>
<span class="c1">// remove these lines:</span>
<span class="k">new</span> <span class="nx">HtmlWebpackPlugin</span><span class="p">({</span>
<span class="na">filename</span><span class="p">:</span> <span class="s1">'index.html'</span><span class="p">,</span>
<span class="na">template</span><span class="p">:</span> <span class="s1">'index.html'</span><span class="p">,</span>
<span class="na">inject</span><span class="p">:</span> <span class="kc">true</span>
<span class="p">}),</span></code></pre></figure>
<p><del>For the same reason we want to remove it from the production config, but we want to keep it in tests. Simply move this bit of conde from <code class="highlighter-rouge">build/webpack.prod.conf.js</code> to <code class="highlighter-rouge">build/webpack.test.conf.js</code></del></p>
<figure class="highlight"><pre><code class="language-javascript" data-lang="javascript"><span class="k">new</span> <span class="nx">HtmlWebpackPlugin</span><span class="p">({</span>
<span class="na">filename</span><span class="p">:</span> <span class="nx">process</span><span class="p">.</span><span class="nx">env</span><span class="p">.</span><span class="nx">NODE_ENV</span> <span class="o">===</span> <span class="s1">'testing'</span>
<span class="p">?</span> <span class="s1">'index.html'</span>
<span class="p">:</span> <span class="nx">config</span><span class="p">.</span><span class="nx">build</span><span class="p">.</span><span class="nx">index</span><span class="p">,</span>
<span class="na">template</span><span class="p">:</span> <span class="s1">'index.html'</span><span class="p">,</span>
<span class="na">inject</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
<span class="na">minify</span><span class="p">:</span> <span class="p">{</span>
<span class="na">removeComments</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
<span class="na">collapseWhitespace</span><span class="p">:</span> <span class="kc">true</span><span class="p">,</span>
<span class="na">removeAttributeQuotes</span><span class="p">:</span> <span class="kc">true</span>
<span class="c1">// more options:</span>
<span class="c1">// https://github.com/kangax/html-minifier#options-quick-reference</span>
<span class="p">},</span>
<span class="c1">// necessary to consistently work with multiple chunks via CommonsChunkPlugin</span>
<span class="na">chunksSortMode</span><span class="p">:</span> <span class="s1">'dependency'</span>
<span class="p">}),</span></code></pre></figure>
<p>And last but not least lets add the following to your <code class="highlighter-rouge">.gitignore</code> file:</p>
<figure class="highlight"><pre><code class="language-git" data-lang="git">webpack-stats.json
public/
dist/</code></pre></figure>
<h1 id="conclusions">Conclusions:</h1>
<p>To have a good dev environment we have used Django as our main dev server, which will in turn proxy any asset request to webpack. This required a good deal of configuration. In doing so we have simplified the development workflow so we can spend more time programming and less time switching between browsers or fine tuning the config for new cases that may appear.</p>
<p>We have also paved the path to production deployment, but that’s something that we will cover in the next post.</p>
<p>Until then,</p>
<p><strong>Happy hacking!</strong></p>
<h4 id="references-used">References used</h4>
<ul>
<li><a href="https://stackoverflow.com/questions/41342144/webpack-hmr-webpack-hmr-404-not-found">https://stackoverflow.com/questions/41342144/webpack-hmr-webpack-hmr-404-not-found</a></li>
<li><a href="https://github.com/webpack/webpack-dev-middleware">https://github.com/webpack/webpack-dev-middleware</a></li>
<li><a href="https://gist.github.com/Belgabor/130e7770575e74581b67597fcb61717e">https://gist.github.com/Belgabor/130e7770575e74581b67597fcb61717e</a></li>
<li><a href="https://github.com/ezhome/django-webpack-loader/tree/master/examples">https://github.com/ezhome/django-webpack-loader/tree/master/examples</a></li>
<li><a href="https://github.com/rokups/hello-vue-django">https://github.com/rokups/hello-vue-django</a></li>
<li><a href="https://gist.github.com/genomics-geek/81c6880ca862d99574c6f84dec81acb0">https://gist.github.com/genomics-geek/81c6880ca862d99574c6f84dec81acb0</a></li>
<li><a href="https://cscheng.info/2016/08/03/integrating-webpack-dev-server-with-django.html">https://cscheng.info/2016/08/03/integrating-webpack-dev-server-with-django.html</a></li>
<li><a href="http://vuejs-templates.github.io/webpack/backend.html">http://vuejs-templates.github.io/webpack/backend.html</a></li>
<li><a href="http://owaislone.org/blog/webpack-plus-reactjs-and-django/">http://owaislone.org/blog/webpack-plus-reactjs-and-django/</a></li>
</ul>
<div style="display:none">
# Getting ready for production
https://docs.djangoproject.com/en/1.11/ref/templates/api/#writing-your-own-context-processors
https://git.embl.de/mainar/my.embo.org/compare/vue-bulma-bootstraping...django-with-vue?view=parallel
</div>
<p><br />
<br /></p>
<div class="hr"></div>
<p><br />
<br />
<script src="https://utteranc.es/client.js" repo="ariera/ariera.github.io" issue-term="pathname" label="comments" theme="github-light" crossorigin="anonymous" async="">
</script></p>UPDATE 2018-11-12: Rodrigo Smaniotto wrote an updated version and more complete than this article using Dajngo2 and Vue CLI3. I can only recommend checking it here: https://medium.com/@rodrigosmaniotto/integrating-django-and-vuejs-with-vue-cli-3-and-webpack-loader-145c3b98501adev-toolbar a web developer’s best friend!2017-09-20T16:05:28+00:002017-09-20T16:05:28+00:00http://ariera.github.io/2017/09/20/dev-toolbar-a-web-developers-best-friend<p>The <strong>dev-toolbar</strong> is a simple concept we came up with many years ago. The idea is to provide developers with a toolbelt full of tricks (shortcuts, toggles, login-as…) hidden in the frontend. It is such an integral part of my workflow that it is usually the first thing I set up in any project.</p>
<p>In this entry we will see how to set it up in a Django web application. We will focus on <strong>the feature that everybody loves</strong>: the ability to login as any user in the database (great time saver). We call it <strong>impersonate</strong> , and this is how it looks like:</p>
<p><img src="http://ariera.github.io/assets/dev-toolbar.png" alt="developer toolbar" /></p>
<p>A simple select with all the users in your database. Click on one and you’ll be logged in as him/her.</p>
<p>Let’s get on with it!</p>
<h3 id="1-create-a-dev-app">1. Create a dev app</h3>
<p>We will start by creating a new app that we will call <code class="highlighter-rouge">dev</code>. It will hold all of our impersonation logic, but is also a useful place to have to keep a style guide for developers, or some handy snippets to share accross the team, or maybe just to house a temporary experiment.</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">python manage.py startapp dev</code></pre></figure>
<p>and don’t forget to add it to your <code class="highlighter-rouge">INSTALLED_APPS</code></p>
<div class="file-title">./your_app/settings.py</div>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./your_app/settings.py</span>
<span class="n">INSTALLED_APPS</span> <span class="o">=</span> <span class="p">[</span>
<span class="s">'dev.apps.DevConfig'</span><span class="p">,</span>
<span class="c"># ...</span>
<span class="p">]</span></code></pre></figure>
<h3 id="2-the-view-and-urls">2. The view and urls</h3>
<p>Here we will control if the current user is allowed to impersonate or not, and perform the actual impersonation. In this case we are allowing any superuser to login as any other user, but you may choose any other criteria, for example only in development environment, or only if <code class="highlighter-rouge">DEBUG</code> is <code class="highlighter-rouge">True</code>.</p>
<div class="file-title">./dev/views.py</div>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./dev/views.py</span>
<span class="kn">from</span> <span class="nn">django.shortcuts</span> <span class="kn">import</span> <span class="n">redirect</span><span class="p">,</span> <span class="n">render</span><span class="p">,</span> <span class="n">get_object_or_404</span>
<span class="kn">from</span> <span class="nn">django.contrib.auth</span> <span class="kn">import</span> <span class="n">get_user_model</span><span class="p">,</span> <span class="n">login</span>
<span class="kn">from</span> <span class="nn">django.http</span> <span class="kn">import</span> <span class="n">Http404</span>
<span class="k">def</span> <span class="nf">impersonate</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">user_id</span><span class="p">):</span>
<span class="n">current_user</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">user</span>
<span class="k">if</span> <span class="n">current_user</span><span class="o">.</span><span class="n">is_superuser</span><span class="p">:</span>
<span class="n">user</span> <span class="o">=</span> <span class="n">get_object_or_404</span><span class="p">(</span><span class="n">get_user_model</span><span class="p">(),</span> <span class="n">pk</span><span class="o">=</span><span class="n">user_id</span><span class="p">)</span>
<span class="n">login</span><span class="p">(</span><span class="n">request</span><span class="p">,</span> <span class="n">user</span><span class="p">)</span>
<span class="k">return</span> <span class="n">redirect</span><span class="p">(</span><span class="s">'/dev'</span><span class="p">)</span>
<span class="k">else</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">Http404</span><span class="p">(</span><span class="s">"Impersonate is only available for superusers"</span><span class="p">)</span></code></pre></figure>
<div class="file-title">./dev/urls.py</div>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./dev/urls.py</span>
<span class="kn">from</span> <span class="nn">django.conf.urls</span> <span class="kn">import</span> <span class="n">url</span>
<span class="kn">from</span> <span class="nn">.</span> <span class="kn">import</span> <span class="n">views</span>
<span class="n">app_name</span> <span class="o">=</span> <span class="s">'dev'</span>
<span class="n">urlpatterns</span> <span class="o">=</span> <span class="p">[</span>
<span class="n">url</span><span class="p">(</span><span class="s">r'^$'</span><span class="p">,</span> <span class="n">views</span><span class="o">.</span><span class="n">index</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s">'index'</span><span class="p">),</span>
<span class="n">url</span><span class="p">(</span><span class="s">r'^impersonate/(?P<user_id>[0-9]+)/$'</span><span class="p">,</span> <span class="n">views</span><span class="o">.</span><span class="n">impersonate</span><span class="p">,</span> <span class="n">name</span><span class="o">=</span><span class="s">'impersonate'</span><span class="p">),</span>
<span class="p">]</span></code></pre></figure>
<div class="file-title">./your_app/urls.py</div>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./your_app/urls.py</span>
<span class="kn">from</span> <span class="nn">django.conf.urls</span> <span class="kn">import</span> <span class="n">include</span><span class="p">,</span> <span class="n">url</span>
<span class="kn">from</span> <span class="nn">django.contrib</span> <span class="kn">import</span> <span class="n">admin</span>
<span class="n">urlpatterns</span> <span class="o">=</span> <span class="p">[</span>
<span class="n">url</span><span class="p">(</span><span class="s">r'^dev/'</span><span class="p">,</span> <span class="n">include</span><span class="p">(</span><span class="s">'dev.urls'</span><span class="p">)),</span>
<span class="n">url</span><span class="p">(</span><span class="s">r'^admin/'</span><span class="p">,</span> <span class="n">admin</span><span class="o">.</span><span class="n">site</span><span class="o">.</span><span class="n">urls</span><span class="p">),</span>
<span class="p">]</span></code></pre></figure>
<h3 id="3-template-tag">3. Template tag</h3>
<p>We want to reuse our toolbar across many templates so we will extract it into it’s own template tag, using <a href="https://docs.djangoproject.com/en/1.11/howto/custom-template-tags/#inclusion-tags">Django’s inclusion-tags</a> functionality.</p>
<p>For that you need to create this directory: <code class="highlighter-rouge">./dev/templatetags</code> and an empty <code class="highlighter-rouge">__init__.py</code> file inside of it.</p>
<p>Inside our templatetags folder we will create the following file, which acts like the view of our new <code class="highlighter-rouge">dev_toolbar</code> templatetag. Notice 2 things:</p>
<ol>
<li>First we are linking it to the template <code class="highlighter-rouge">dev/toolbar.html</code> that we will define later.</li>
<li>And second we use <code class="highlighter-rouge">takes_context=True</code> to be able to access to the current user. This will allow us to do some basic authorization (ie. only superusers are allowed to use this feature). This is redundant with the logic we defined before, but it’s good to have a safety net when you’re handling delicate data.</li>
</ol>
<div class="file-title">./dev/templatetags/dev_toolbar.py</div>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./dev/templatetags/dev_toolbar.py</span>
<span class="kn">from</span> <span class="nn">django.contrib.auth</span> <span class="kn">import</span> <span class="n">get_user_model</span>
<span class="kn">from</span> <span class="nn">django</span> <span class="kn">import</span> <span class="n">template</span>
<span class="n">register</span> <span class="o">=</span> <span class="n">template</span><span class="o">.</span><span class="n">Library</span><span class="p">()</span>
<span class="nd">@register.inclusion_tag</span><span class="p">(</span><span class="s">'dev/toolbar.html'</span><span class="p">,</span> <span class="n">takes_context</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="k">def</span> <span class="nf">dev_toolbar</span><span class="p">(</span><span class="n">context</span><span class="p">):</span>
<span class="n">request</span> <span class="o">=</span> <span class="n">context</span><span class="p">[</span><span class="s">'request'</span><span class="p">]</span>
<span class="n">current_user</span> <span class="o">=</span> <span class="n">request</span><span class="o">.</span><span class="n">user</span>
<span class="k">if</span> <span class="n">current_user</span><span class="o">.</span><span class="n">is_superuser</span><span class="p">:</span>
<span class="n">users</span> <span class="o">=</span> <span class="n">get_user_model</span><span class="p">()</span><span class="o">.</span><span class="n">objects</span><span class="o">.</span><span class="nb">all</span><span class="p">()</span>
<span class="k">else</span><span class="p">:</span>
<span class="n">users</span> <span class="o">=</span> <span class="p">[]</span>
<span class="k">return</span> <span class="p">{</span><span class="s">'users'</span><span class="p">:</span> <span class="n">users</span><span class="p">,</span> <span class="s">'current_user'</span><span class="p">:</span> <span class="n">current_user</span><span class="p">}</span></code></pre></figure>
<p>The template file contains its own css, javascript and html. Some simple positioning and styling and bit of css+js logic to make the bar more subtle.</p>
<p>We will render a select box with all the users in the database and a javascript will take care of calling the impersonate action whenever we choose another user other than ourselves. Two more handy links to <code class="highlighter-rouge">/dev</code> and <code class="highlighter-rouge">/admin</code> areas are included.</p>
<p>Last, but not least, I usually include a small close button, it comes in handy sometimes when doing CSS work, you just want to get rid of the bar.</p>
<div class="file-title">./dev/templates/dev/toolbar.html</div>
<figure class="highlight"><pre><code class="language-html" data-lang="html"><span class="c"><!-- ./dev/templates/dev/toolbar.html --></span>
{% if current_user.is_superuser %}
<span class="nt"><style></span>
<span class="nf">#dev-toolbar</span><span class="p">{</span>
<span class="nl">background-color</span><span class="p">:</span> <span class="m">#ccc</span><span class="p">;</span>
<span class="nl">position</span><span class="p">:</span> <span class="nb">fixed</span><span class="p">;</span>
<span class="nl">bottom</span><span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="nl">right</span><span class="p">:</span> <span class="m">0</span><span class="p">;</span>
<span class="nl">padding</span><span class="p">:</span> <span class="m">5px</span><span class="p">;</span>
<span class="p">}</span>
<span class="nf">#dev-toolbar</span><span class="nc">.subtle</span> <span class="p">{</span>
<span class="nl">opacity</span><span class="p">:</span> <span class="m">0.2</span><span class="p">;</span>
<span class="p">}</span>
<span class="nt"></style></span>
<span class="nt"><script></span>
<span class="nb">window</span><span class="p">.</span><span class="nx">dev</span> <span class="o">=</span> <span class="p">{</span>
<span class="na">closeToolbar</span><span class="p">:</span> <span class="kd">function</span> <span class="nx">closeToolbar</span><span class="p">(){</span>
<span class="kd">var</span> <span class="nx">toolbar</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">getElementById</span><span class="p">(</span><span class="s2">"dev-toolbar"</span><span class="p">)</span>
<span class="nx">toolbar</span><span class="p">.</span><span class="nx">parentElement</span><span class="p">.</span><span class="nx">removeChild</span><span class="p">(</span><span class="nx">toolbar</span><span class="p">)</span>
<span class="p">},</span>
<span class="na">showToolbar</span><span class="p">:</span> <span class="kd">function</span> <span class="nx">showToolbar</span><span class="p">(){</span>
<span class="kd">var</span> <span class="nx">toolbar</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">getElementById</span><span class="p">(</span><span class="s2">"dev-toolbar"</span><span class="p">)</span>
<span class="nx">toolbar</span><span class="p">.</span><span class="nx">classList</span><span class="p">.</span><span class="nx">remove</span><span class="p">(</span><span class="s2">"subtle"</span><span class="p">);</span>
<span class="p">},</span>
<span class="na">impersonate</span><span class="p">:</span> <span class="kd">function</span> <span class="nx">impersonate</span><span class="p">()</span> <span class="p">{</span>
<span class="kd">var</span> <span class="nx">select</span> <span class="o">=</span> <span class="nb">document</span><span class="p">.</span><span class="nx">getElementById</span><span class="p">(</span><span class="s2">"dev-impersonate"</span><span class="p">)</span>
<span class="nb">window</span><span class="p">.</span><span class="nx">location</span> <span class="o">=</span> <span class="nx">select</span><span class="p">.</span><span class="nx">value</span>
<span class="p">}</span>
<span class="p">}</span>
<span class="nt"></script></span>
<span class="nt"><div</span> <span class="na">id=</span><span class="s">"dev-toolbar"</span> <span class="na">class=</span><span class="s">"subtle"</span> <span class="na">onmouseover=</span><span class="s">"dev.showToolbar()"</span><span class="nt">></span>
<span class="nt"><small><a</span> <span class="na">href=</span><span class="s">"/dev"</span><span class="nt">></span>dev<span class="nt"></a></small></span>
<span class="nt"><small><a</span> <span class="na">href=</span><span class="s">"/admin"</span><span class="nt">></span>admin<span class="nt"></a></small></span>
{% if users %}
<span class="nt"><select</span> <span class="na">id=</span><span class="s">"dev-impersonate"</span> <span class="na">onchange=</span><span class="s">"dev.impersonate()"</span><span class="nt">></span>
{% for user in users %}
<span class="nt"><option</span> <span class="na">value=</span><span class="s">"{% url 'dev:impersonate' user.id %}"</span>
<span class="err">{%</span> <span class="na">if</span> <span class="na">user =</span><span class="s">=</span> <span class="na">current_user</span> <span class="err">%}</span><span class="na">selected=</span><span class="s">"selected"</span><span class="err">{%</span> <span class="na">endif</span> <span class="err">%}</span><span class="nt">></span>
{{user.email}}
<span class="nt"></option></span>
{% endfor %}
<span class="nt"></select></span>
{% endif %}
<span class="nt"><button</span> <span class="na">id=</span><span class="s">"dev-close-toolbar"</span> <span class="na">onclick=</span><span class="s">"dev.closeToolbar()"</span><span class="nt">></span>X<span class="nt"></button></span>
<span class="nt"></div></span>
{% endif %}</code></pre></figure>
<h3 id="4-using-it">4. Using it</h3>
<p>Our feature is complete and ready to be used. Simply load it in any view and call it.</p>
<div class="highlighter-rouge"><div class="highlight"><pre class="highlight"><code>{% load dev_toolbar %}
{% dev_toolbar %}
</code></pre></div></div>
<h3 id="5-show-it-in-the-admin-panel">5. Show it in the admin panel</h3>
<p>We are going to override the default <code class="highlighter-rouge">admin/base.html</code> template and include our new <code class="highlighter-rouge">dev_toolbar</code> snippet. For that we need to create a generic <code class="highlighter-rouge">./templates</code> directory at the root of the project and add it to <code class="highlighter-rouge">TEMPLATES['DIRS']</code> in the settings of the project.</p>
<div class="file-title">./your_app/settings.py</div>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./your_app/settings.py</span>
<span class="n">TEMPLATES</span> <span class="o">=</span> <span class="p">[</span>
<span class="p">{</span>
<span class="s">'DIRS'</span><span class="p">:</span> <span class="p">[</span><span class="n">os</span><span class="o">.</span><span class="n">path</span><span class="o">.</span><span class="n">join</span><span class="p">(</span><span class="n">BASE_DIR</span><span class="p">,</span> <span class="s">'templates'</span><span class="p">)],</span>
<span class="c"># ...</span>
<span class="p">}</span>
<span class="p">]</span></code></pre></figure>
<p>And now create a new file to override the default administration base tempalte.</p>
<div class="file-title">./templates/admin/base.html</div>
<figure class="highlight"><pre><code class="language-html" data-lang="html"> <span class="c"><!-- ./templates/admin/base.html --></span>
{% extends "admin/base.html" %}
{% load dev_toolbar %}
{% block footer %}
{{block.super}}
{% dev_toolbar %}
{% endblock %}</code></pre></figure>
<h4 id="references">References</h4>
<ul>
<li><a href="https://docs.djangoproject.com/en/1.11/topics/http/views/#the-http404-exception">Django Http404</a></li>
<li><a href="https://docs.djangoproject.com/en/1.11/howto/custom-template-tags/#inclusion-tags">Django inclusion-tags</a></li>
<li><a href="https://stackoverflow.com/questions/6713022/handling-request-in-django-inclusion-template-tag">SO: Handling request in django inclusion template tag</a></li>
<li><a href="https://stackoverflow.com/a/6586068">SO: How to override and extend basic Django admin templates?</a></li>
</ul>The dev-toolbar is a simple concept we came up with many years ago. The idea is to provide developers with a toolbelt full of tricks (shortcuts, toggles, login-as…) hidden in the frontend. It is such an integral part of my workflow that it is usually the first thing I set up in any project.How to use email as user identifier in Django 1.112017-09-20T08:30:10+00:002017-09-20T08:30:10+00:00http://ariera.github.io/2017/09/20/how-to-use-email-as-user-identifier-in-django-1-11<p>I found several guides and tips on how to do this. They were all very inspiring and helpful but I had to figure some things on my own because none of them provided all I needed: a fully working admin panel that replicates all the functionality of the original one.</p>
<p>I won’t stop to explain what every line is doing, in the end I’m just trying to save myself some trouble in the future, but I share it because it may be useful to someone else as well. At the moment of writing this I am using <code class="highlighter-rouge">Django==1.11.5</code>.</p>
<h4 id="before-we-start">Before we start</h4>
<p>Remember that you need to do this at the beginning of the life of your project, before running any migrations, so drop&create your DB if you are already that far. I would do this in my local machine with postgres:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">pg_ctl <span class="nt">-D</span> /usr/local/var/postgres restart
dropdb my_app
createdb my_app</code></pre></figure>
<h4 id="first-create-your-new-app">First create your new app</h4>
<p>Since it this is gonna be very generic, project-wide app, I like to call it <code class="highlighter-rouge">base</code>. I may end up placing other basic models / views in here.</p>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="n">python</span> <span class="n">manage</span><span class="o">.</span><span class="n">py</span> <span class="n">startapp</span> <span class="n">base</span></code></pre></figure>
<h4 id="configure-your-settings">Configure your settings</h4>
<div class="file-title">./my_app/settings.py</div>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="c"># ./my_app/settings.py</span>
<span class="n">AUTH_USER_MODEL</span> <span class="o">=</span> <span class="s">'base.User'</span>
<span class="n">INSTALLED_APPS</span> <span class="o">=</span> <span class="p">[</span>
<span class="s">'base.apps.BaseConfig'</span><span class="p">,</span>
<span class="c"># ...</span>
<span class="p">]</span></code></pre></figure>
<h4 id="configure-your-models-and-admin">Configure your <code class="highlighter-rouge">models</code> and <code class="highlighter-rouge">admin</code></h4>
<p>This will be your models, highly inspired by the <a href="https://github.com/django/django/blob/a96b981d84367fd41b1df40adf3ac9ca71a741dd/django/contrib/auth/models.py#L288">original Django source code</a>.</p>
<div class="file-title">./base/models.py</div>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="kn">from</span> <span class="nn">django.db</span> <span class="kn">import</span> <span class="n">models</span>
<span class="kn">from</span> <span class="nn">django.contrib.auth.models</span> <span class="kn">import</span> <span class="n">AbstractBaseUser</span>
<span class="kn">from</span> <span class="nn">django.contrib.auth.models</span> <span class="kn">import</span> <span class="n">BaseUserManager</span>
<span class="kn">from</span> <span class="nn">django.contrib.auth.models</span> <span class="kn">import</span> <span class="n">PermissionsMixin</span>
<span class="kn">from</span> <span class="nn">django.utils.translation</span> <span class="kn">import</span> <span class="n">ugettext_lazy</span> <span class="k">as</span> <span class="n">_</span>
<span class="k">class</span> <span class="nc">MyUserManager</span><span class="p">(</span><span class="n">BaseUserManager</span><span class="p">):</span>
<span class="s">"""
A custom user manager to deal with emails as unique identifiers for auth
instead of usernames. The default that's used is "UserManager"
"""</span>
<span class="k">def</span> <span class="nf">_create_user</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">email</span><span class="p">,</span> <span class="n">password</span><span class="p">,</span> <span class="o">**</span><span class="n">extra_fields</span><span class="p">):</span>
<span class="s">"""
Creates and saves a User with the given email and password.
"""</span>
<span class="k">if</span> <span class="ow">not</span> <span class="n">email</span><span class="p">:</span>
<span class="k">raise</span> <span class="nb">ValueError</span><span class="p">(</span><span class="s">'The Email must be set'</span><span class="p">)</span>
<span class="n">email</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">normalize_email</span><span class="p">(</span><span class="n">email</span><span class="p">)</span>
<span class="n">user</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">model</span><span class="p">(</span><span class="n">email</span><span class="o">=</span><span class="n">email</span><span class="p">,</span> <span class="o">**</span><span class="n">extra_fields</span><span class="p">)</span>
<span class="n">user</span><span class="o">.</span><span class="n">set_password</span><span class="p">(</span><span class="n">password</span><span class="p">)</span>
<span class="n">user</span><span class="o">.</span><span class="n">save</span><span class="p">()</span>
<span class="k">return</span> <span class="n">user</span>
<span class="k">def</span> <span class="nf">create_superuser</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">email</span><span class="p">,</span> <span class="n">password</span><span class="p">,</span> <span class="o">**</span><span class="n">extra_fields</span><span class="p">):</span>
<span class="n">extra_fields</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="s">'is_staff'</span><span class="p">,</span> <span class="bp">True</span><span class="p">)</span>
<span class="n">extra_fields</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="s">'is_superuser'</span><span class="p">,</span> <span class="bp">True</span><span class="p">)</span>
<span class="n">extra_fields</span><span class="o">.</span><span class="n">setdefault</span><span class="p">(</span><span class="s">'is_active'</span><span class="p">,</span> <span class="bp">True</span><span class="p">)</span>
<span class="k">if</span> <span class="n">extra_fields</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'is_staff'</span><span class="p">)</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">True</span><span class="p">:</span>
<span class="k">raise</span> <span class="nb">ValueError</span><span class="p">(</span><span class="s">'Superuser must have is_staff=True.'</span><span class="p">)</span>
<span class="k">if</span> <span class="n">extra_fields</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">'is_superuser'</span><span class="p">)</span> <span class="ow">is</span> <span class="ow">not</span> <span class="bp">True</span><span class="p">:</span>
<span class="k">raise</span> <span class="nb">ValueError</span><span class="p">(</span><span class="s">'Superuser must have is_superuser=True.'</span><span class="p">)</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">_create_user</span><span class="p">(</span><span class="n">email</span><span class="p">,</span> <span class="n">password</span><span class="p">,</span> <span class="o">**</span><span class="n">extra_fields</span><span class="p">)</span>
<span class="k">class</span> <span class="nc">User</span><span class="p">(</span><span class="n">AbstractBaseUser</span><span class="p">,</span> <span class="n">PermissionsMixin</span><span class="p">):</span>
<span class="n">email</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">EmailField</span><span class="p">(</span><span class="n">unique</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span> <span class="n">null</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">is_staff</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">BooleanField</span><span class="p">(</span>
<span class="n">_</span><span class="p">(</span><span class="s">'staff status'</span><span class="p">),</span>
<span class="n">default</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span>
<span class="n">help_text</span><span class="o">=</span><span class="n">_</span><span class="p">(</span><span class="s">'Designates whether the user can log into this site.'</span><span class="p">),</span>
<span class="p">)</span>
<span class="n">is_active</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">BooleanField</span><span class="p">(</span>
<span class="n">_</span><span class="p">(</span><span class="s">'active'</span><span class="p">),</span>
<span class="n">default</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span>
<span class="n">help_text</span><span class="o">=</span><span class="n">_</span><span class="p">(</span>
<span class="s">'Designates whether this user should be treated as active. '</span>
<span class="s">'Unselect this instead of deleting accounts.'</span>
<span class="p">),</span>
<span class="p">)</span>
<span class="n">is_superuser</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">BooleanField</span><span class="p">(</span><span class="n">default</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span> <span class="n">help_text</span><span class="o">=</span><span class="s">'Designates that this user has all permissions without explicitly assigning them.'</span><span class="p">,</span> <span class="n">verbose_name</span><span class="o">=</span><span class="s">'superuser status'</span><span class="p">)</span>
<span class="n">created_at</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">DateTimeField</span><span class="p">(</span><span class="n">auto_now_add</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">updated_at</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">DateTimeField</span><span class="p">(</span><span class="n">auto_now</span><span class="o">=</span><span class="bp">True</span><span class="p">)</span>
<span class="n">first_name</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">CharField</span><span class="p">(</span><span class="n">blank</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span> <span class="n">max_length</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">verbose_name</span><span class="o">=</span><span class="s">'first name'</span><span class="p">)</span>
<span class="n">last_name</span> <span class="o">=</span> <span class="n">models</span><span class="o">.</span><span class="n">CharField</span><span class="p">(</span><span class="n">blank</span><span class="o">=</span><span class="bp">True</span><span class="p">,</span> <span class="n">max_length</span><span class="o">=</span><span class="mi">30</span><span class="p">,</span> <span class="n">verbose_name</span><span class="o">=</span><span class="s">'last name'</span><span class="p">)</span>
<span class="n">USERNAME_FIELD</span> <span class="o">=</span> <span class="s">'email'</span>
<span class="n">objects</span> <span class="o">=</span> <span class="n">MyUserManager</span><span class="p">()</span>
<span class="k">def</span> <span class="nf">__str__</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">email</span>
<span class="k">def</span> <span class="nf">get_full_name</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">email</span>
<span class="k">def</span> <span class="nf">get_short_name</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="k">return</span> <span class="bp">self</span><span class="o">.</span><span class="n">email</span></code></pre></figure>
<p>And here is the admin file, also highly inspired by the <a href="https://github.com/django/django/blob/a96b981d84367fd41b1df40adf3ac9ca71a741dd/django/contrib/auth/admin.py#L42">original Django source code</a></p>
<div class="file-title">./base/admin.py</div>
<figure class="highlight"><pre><code class="language-python" data-lang="python"><span class="kn">from</span> <span class="nn">django</span> <span class="kn">import</span> <span class="n">forms</span>
<span class="kn">from</span> <span class="nn">django.contrib</span> <span class="kn">import</span> <span class="n">admin</span>
<span class="kn">from</span> <span class="nn">django.contrib.auth</span> <span class="kn">import</span> <span class="n">password_validation</span>
<span class="kn">from</span> <span class="nn">django.contrib.auth.admin</span> <span class="kn">import</span> <span class="n">UserAdmin</span> <span class="k">as</span> <span class="n">BaseUserAdmin</span>
<span class="kn">from</span> <span class="nn">django.contrib.auth.forms</span> <span class="kn">import</span> <span class="n">ReadOnlyPasswordHashField</span>
<span class="kn">from</span> <span class="nn">django.contrib.auth.forms</span> <span class="kn">import</span> <span class="n">UserChangeForm</span> <span class="k">as</span> <span class="n">BaseUserChangeForm</span>
<span class="kn">from</span> <span class="nn">django.utils.translation</span> <span class="kn">import</span> <span class="n">gettext</span><span class="p">,</span> <span class="n">gettext_lazy</span> <span class="k">as</span> <span class="n">_</span>
<span class="kn">from</span> <span class="nn">.models</span> <span class="kn">import</span> <span class="n">User</span>
<span class="k">class</span> <span class="nc">UserCreationForm</span><span class="p">(</span><span class="n">forms</span><span class="o">.</span><span class="n">ModelForm</span><span class="p">):</span>
<span class="s">"""A form for creating new users. Includes all the required
fields, plus a repeated password."""</span>
<span class="n">password1</span> <span class="o">=</span> <span class="n">forms</span><span class="o">.</span><span class="n">CharField</span><span class="p">(</span>
<span class="n">label</span><span class="o">=</span><span class="n">_</span><span class="p">(</span><span class="s">"Password"</span><span class="p">),</span>
<span class="n">strip</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span>
<span class="n">widget</span><span class="o">=</span><span class="n">forms</span><span class="o">.</span><span class="n">PasswordInput</span><span class="p">,</span>
<span class="n">help_text</span><span class="o">=</span><span class="n">password_validation</span><span class="o">.</span><span class="n">password_validators_help_text_html</span><span class="p">(),</span>
<span class="p">)</span>
<span class="n">password2</span> <span class="o">=</span> <span class="n">forms</span><span class="o">.</span><span class="n">CharField</span><span class="p">(</span>
<span class="n">label</span><span class="o">=</span><span class="n">_</span><span class="p">(</span><span class="s">"Password confirmation"</span><span class="p">),</span>
<span class="n">widget</span><span class="o">=</span><span class="n">forms</span><span class="o">.</span><span class="n">PasswordInput</span><span class="p">,</span>
<span class="n">strip</span><span class="o">=</span><span class="bp">False</span><span class="p">,</span>
<span class="n">help_text</span><span class="o">=</span><span class="n">_</span><span class="p">(</span><span class="s">"Enter the same password as before, for verification."</span><span class="p">),</span>
<span class="p">)</span>
<span class="k">class</span> <span class="nc">Meta</span><span class="p">:</span>
<span class="n">model</span> <span class="o">=</span> <span class="n">User</span>
<span class="n">fields</span> <span class="o">=</span> <span class="p">(</span><span class="s">'email'</span><span class="p">,</span> <span class="s">'is_staff'</span><span class="p">,</span> <span class="s">'is_active'</span><span class="p">,</span> <span class="s">'first_name'</span><span class="p">,</span> <span class="s">'last_name'</span><span class="p">,</span> <span class="p">)</span>
<span class="k">def</span> <span class="nf">clean_password2</span><span class="p">(</span><span class="bp">self</span><span class="p">):</span>
<span class="c"># Check that the two password entries match</span>
<span class="n">password1</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">cleaned_data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"password1"</span><span class="p">)</span>
<span class="n">password2</span> <span class="o">=</span> <span class="bp">self</span><span class="o">.</span><span class="n">cleaned_data</span><span class="o">.</span><span class="n">get</span><span class="p">(</span><span class="s">"password2"</span><span class="p">)</span>
<span class="k">if</span> <span class="n">password1</span> <span class="ow">and</span> <span class="n">password2</span> <span class="ow">and</span> <span class="n">password1</span> <span class="o">!=</span> <span class="n">password2</span><span class="p">:</span>
<span class="k">raise</span> <span class="n">forms</span><span class="o">.</span><span class="n">ValidationError</span><span class="p">(</span><span class="s">"Passwords don't match"</span><span class="p">)</span>
<span class="k">return</span> <span class="n">password2</span>
<span class="k">def</span> <span class="nf">save</span><span class="p">(</span><span class="bp">self</span><span class="p">,</span> <span class="n">commit</span><span class="o">=</span><span class="bp">True</span><span class="p">):</span>
<span class="c"># Save the provided password in hashed format</span>
<span class="n">user</span> <span class="o">=</span> <span class="nb">super</span><span class="p">()</span><span class="o">.</span><span class="n">save</span><span class="p">(</span><span class="n">commit</span><span class="o">=</span><span class="bp">False</span><span class="p">)</span>
<span class="n">user</span><span class="o">.</span><span class="n">set_password</span><span class="p">(</span><span class="bp">self</span><span class="o">.</span><span class="n">cleaned_data</span><span class="p">[</span><span class="s">"password1"</span><span class="p">])</span>
<span class="k">if</span> <span class="n">commit</span><span class="p">:</span>
<span class="n">user</span><span class="o">.</span><span class="n">save</span><span class="p">()</span>
<span class="k">return</span> <span class="n">user</span>
<span class="k">class</span> <span class="nc">UserChangeForm</span><span class="p">(</span><span class="n">BaseUserChangeForm</span><span class="p">):</span>
<span class="s">"""
Override as needed
"""</span>
<span class="k">class</span> <span class="nc">UserAdmin</span><span class="p">(</span><span class="n">BaseUserAdmin</span><span class="p">):</span>
<span class="c"># The forms to add and change user instances</span>
<span class="n">form</span> <span class="o">=</span> <span class="n">UserChangeForm</span>
<span class="n">add_form</span> <span class="o">=</span> <span class="n">UserCreationForm</span>
<span class="c"># The fields to be used in displaying the User model.</span>
<span class="c"># These override the definitions on the base UserAdmin</span>
<span class="c"># that reference specific fields on auth.User.</span>
<span class="n">list_display</span> <span class="o">=</span> <span class="p">(</span><span class="s">'email'</span><span class="p">,</span> <span class="s">'is_superuser'</span><span class="p">,</span> <span class="s">'is_staff'</span><span class="p">,</span> <span class="s">'is_active'</span><span class="p">,</span> <span class="s">'first_name'</span><span class="p">,</span> <span class="s">'last_name'</span><span class="p">,</span> <span class="s">'created_at'</span><span class="p">,</span> <span class="s">'updated_at'</span><span class="p">,</span> <span class="s">'last_login'</span><span class="p">)</span>
<span class="c"># list_filter = ('email',)</span>
<span class="n">readonly_fields</span><span class="o">=</span><span class="p">(</span><span class="s">'created_at'</span><span class="p">,</span> <span class="s">'updated_at'</span><span class="p">,)</span>
<span class="n">fieldsets</span> <span class="o">=</span> <span class="p">(</span>
<span class="p">(</span><span class="bp">None</span> <span class="p">,</span> <span class="p">{</span><span class="s">'fields'</span><span class="p">:</span> <span class="p">(</span><span class="s">'email'</span><span class="p">,</span> <span class="s">'password'</span><span class="p">)}),</span>
<span class="p">(</span><span class="n">_</span><span class="p">(</span><span class="s">'Personal info'</span><span class="p">)</span> <span class="p">,</span> <span class="p">{</span><span class="s">'fields'</span><span class="p">:</span> <span class="p">(</span><span class="s">'first_name'</span><span class="p">,</span> <span class="s">'last_name'</span><span class="p">,)}),</span>
<span class="p">(</span><span class="n">_</span><span class="p">(</span><span class="s">'Permissions'</span><span class="p">)</span> <span class="p">,</span> <span class="p">{</span><span class="s">'fields'</span><span class="p">:</span> <span class="p">(</span><span class="s">'is_active'</span><span class="p">,</span> <span class="s">'is_staff'</span><span class="p">,</span> <span class="s">'is_superuser'</span><span class="p">,</span> <span class="s">'groups'</span><span class="p">,</span> <span class="s">'user_permissions'</span><span class="p">)}),</span>
<span class="p">(</span><span class="n">_</span><span class="p">(</span><span class="s">'Important dates'</span><span class="p">),</span> <span class="p">{</span><span class="s">'fields'</span><span class="p">:</span> <span class="p">(</span><span class="s">'last_login'</span><span class="p">,</span> <span class="s">'created_at'</span><span class="p">,</span> <span class="s">'updated_at'</span><span class="p">)}),</span>
<span class="p">)</span>
<span class="c"># add_fieldsets is not a standard ModelAdmin attribute. UserAdmin</span>
<span class="c"># overrides get_fieldsets to use this attribute when creating a user.</span>
<span class="n">add_fieldsets</span> <span class="o">=</span> <span class="p">(</span>
<span class="p">(</span><span class="bp">None</span><span class="p">,</span> <span class="p">{</span>
<span class="s">'classes'</span><span class="p">:</span> <span class="p">(</span><span class="s">'wide'</span><span class="p">,),</span>
<span class="s">'fields'</span><span class="p">:</span> <span class="p">(</span><span class="s">'email'</span><span class="p">,</span> <span class="s">'password1'</span><span class="p">,</span> <span class="s">'password2'</span><span class="p">)}</span>
<span class="p">),</span>
<span class="p">)</span>
<span class="n">search_fields</span> <span class="o">=</span> <span class="p">(</span><span class="s">'email'</span><span class="p">,)</span>
<span class="n">ordering</span> <span class="o">=</span> <span class="p">(</span><span class="s">'email'</span><span class="p">,)</span>
<span class="c"># filter_horizontal = ()</span>
<span class="n">admin</span><span class="o">.</span><span class="n">site</span><span class="o">.</span><span class="n">register</span><span class="p">(</span><span class="n">User</span><span class="p">,</span> <span class="n">UserAdmin</span><span class="p">)</span></code></pre></figure>
<h4 id="apply-changes">Apply changes</h4>
<p>And lastly you need to generate your migrations and migrate</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">python manage.py makemigrations base
python manage.py migrate
python manage.py createsuperuser</code></pre></figure>
<h4 id="references">References</h4>
<ul>
<li><a href="https://docs.djangoproject.com/en/1.11/topics/auth/customizing/#substituting-a-custom-user-model">https://docs.djangoproject.com/en/1.11/topics/auth/customizing/#substituting-a-custom-user-model</a></li>
<li><a href="https://docs.djangoproject.com/en/1.11/topics/auth/customizing/#a-full-example">https://docs.djangoproject.com/en/1.11/topics/auth/customizing/#a-full-example</a></li>
<li><a href="https://medium.com/@ramykhuffash/django-authentication-with-just-an-email-and-password-no-username-required-33e47976b517">https://medium.com/@ramykhuffash/django-authentication-with-just-an-email-and-password-no-username-required-33e47976b517</a></li>
<li><a href="https://stackoverflow.com/questions/3967644/django-admin-how-to-display-a-field-that-is-marked-as-editable-false-in-the-mo/3967891#3967891">https://stackoverflow.com/questions/3967644/django-admin-how-to-display-a-field-that-is-marked-as-editable-false-in-the-mo/3967891#3967891</a></li>
</ul>I found several guides and tips on how to do this. They were all very inspiring and helpful but I had to figure some things on my own because none of them provided all I needed: a fully working admin panel that replicates all the functionality of the original one.Setting up mosh on macOS with fish shell2017-03-27T13:34:38+00:002017-03-27T13:34:38+00:00http://ariera.github.io/2017/03/27/setting-up-mosh-on-macos-with-fish-shell<p>It is actually very easy to install both in macOS and Ubuntu.</p>
<p>On mac you just have to do</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">brew install mobile-shell</code></pre></figure>
<p>and on Ubuntu</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>apt-get install mosh</code></pre></figure>
<h4 id="the-problem">The problem</h4>
<p>However, when trying to connect to my server <code class="highlighter-rouge">mosh USER@SERVER</code> I was getting this error</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash">The locale requested by <span class="nv">LC_CTYPE</span><span class="o">=</span>UTF-8 isn<span class="s1">'t available here.
Running `locale-gen UTF-8'</span> may be necessary.
mosh-server needs a UTF-8 native locale to run.
Unfortunately, the <span class="nb">local </span>environment <span class="o">([</span>no charset variables]<span class="o">)</span> specifies
the character <span class="nb">set</span> <span class="s2">"US-ASCII"</span>,
The client-supplied environment <span class="o">(</span><span class="nv">LC_CTYPE</span><span class="o">=</span>UTF-8<span class="o">)</span> specifies
the character <span class="nb">set</span> <span class="s2">"US-ASCII"</span><span class="nb">.</span>
locale: Cannot <span class="nb">set </span>LC_CTYPE to default locale: No such file or directory
locale: Cannot <span class="nb">set </span>LC_ALL to default locale: No such file or directory
<span class="nv">LANG</span><span class="o">=</span>
<span class="nv">LANGUAGE</span><span class="o">=</span>
<span class="nv">LC_CTYPE</span><span class="o">=</span>UTF-8
<span class="nv">LC_NUMERIC</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_TIME</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_COLLATE</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_MONETARY</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_MESSAGES</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_PAPER</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_NAME</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_ADDRESS</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_TELEPHONE</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_MEASUREMENT</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_IDENTIFICATION</span><span class="o">=</span><span class="s2">"POSIX"</span>
<span class="nv">LC_ALL</span><span class="o">=</span>
Connection to embodev2.embo.org closed.
/usr/local/bin/mosh: Did not find mosh server startup message. <span class="o">(</span>Have you installed mosh on your server?<span class="o">)</span></code></pre></figure>
<h4 id="the-solution">The solution</h4>
<p>The solution was as simple as setting your default <code class="highlighter-rouge">LC_ALL</code> env. Simply add this to your to your <code class="highlighter-rouge">~/.config/fish/config.fish</code></p>
<div class="file-title">~/.config/fish/config.fish</div>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">set</span> <span class="nt">-x</span> LC_ALL en_GB.UTF-8</code></pre></figure>
<p>And make sure your server also has that locale availabe:</p>
<figure class="highlight"><pre><code class="language-bash" data-lang="bash"><span class="nb">sudo </span>locale-gen en_GB.utf-8</code></pre></figure>It is actually very easy to install both in macOS and Ubuntu.Setting up GitLab’s Continuous Integration with Laravel, PostgreSQL and PHPUnit2017-03-09T17:06:56+00:002017-03-09T17:06:56+00:00http://ariera.github.io/2017/03/09/settingup-gitlab-ci-with-laravel-postgres-phpunit<p>I recently set up a continuous integration process based on GitLab for a project based on Laravel and PostgreSQL. It took a while to get it right, so I am sharing here my final config.</p>
<h2 id="basic-structure">Basic structure</h2>
<p>The main GitLab CI configuration is held by a file at the root of your project that should be called <code class="highlighter-rouge">.gitlab-ci.yml</code>. From this file it is possible to call other shell scripts, but I have tried to hold up everything in just one place.</p>
<p>We will however manage a second file called <code class="highlighter-rouge">.env.gitlab-ci</code> that will hold all ENV variable configuration optios specific to GitLab CI.</p>
<p>Last but not least you will need a PHPUnit xml configuration file.</p>
<h2 id="step-by-step-look-into-gitlab-ciyml">Step by step look into <code class="highlighter-rouge">.gitlab-ci.yml</code></h2>
<p>We’ll go line by line in the config file, but you can see the final result at the end of the article.</p>
<p> </p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">image</span><span class="pi">:</span> <span class="s">php:7.0</span></code></pre></figure>
<p>Here we select one of the available docker images with php already configured. You can find a complete list here: from https://hub.docker.com/r/_/php/</p>
<p> </p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">services</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">postgres:latest</span></code></pre></figure>
<p>From GitLab’s official documentation <a href="https://docs.gitlab.com/ce/ci/docker/using_docker_images.html#what-is-a-service">src</a>:</p>
<blockquote>
<p>The <code class="highlighter-rouge">services</code> keyword defines just another docker image that is run during your job and is linked to the docker image that the <code class="highlighter-rouge">image</code> keyword defines. This allows you to access the service image during build time.</p>
</blockquote>
<p>It is <strong>very important</strong> to remark that if you add <code class="highlighter-rouge">postgres</code> as service to your application, the service container for PostgresSQL will be accessible under the hostname <code class="highlighter-rouge">postgres</code>. So, in order to access your database service you have to connect to the host named <code class="highlighter-rouge">postgres</code> instead of a socket or <code class="highlighter-rouge">localhost</code>. This will be important later when we configure our <code class="highlighter-rouge">.env.gitlab-ci</code> file.</p>
<p> </p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">variables</span><span class="pi">:</span>
<span class="na">POSTGRES_DB</span><span class="pi">:</span> <span class="s">mydb-test</span>
<span class="na">POSTGRES_USER</span><span class="pi">:</span> <span class="s">runner</span>
<span class="na">POSTGRES_PASSWORD</span><span class="pi">:</span> <span class="s2">"</span><span class="s">"</span></code></pre></figure>
<p>Here you may use all the values you want as long as they match with those of <code class="highlighter-rouge">.env.gitlab-ci</code> and your <code class="highlighter-rouge">phpunit.xml</code>.</p>
<p> </p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">before_script</span><span class="pi">:</span></code></pre></figure>
<p>Again from the official documentation <a href="https://docs.gitlab.com/ce/ci/yaml/README.html#before_script">src</a></p>
<blockquote>
<p><code class="highlighter-rouge">before_script</code> is used to define the command that should be run before all jobs, including deploy jobs, but after the restoration of artifacts. This can be an array or a multi-line string.</p>
</blockquote>
<p>Lets explore all these commands one by one.</p>
<p> </p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="pi">-</span> <span class="pi">></span>
<span class="no">set -xe</span>
<span class="no">&& apt-get update -yqq</span>
<span class="no">&& apt-get install -yqq</span>
<span class="no">git</span>
<span class="no">libicu-dev</span>
<span class="no">libpq-dev</span>
<span class="no">libzip-dev</span>
<span class="no">zlib1g-dev</span></code></pre></figure>
<p>Simply installing some dependencies</p>
<ul>
<li><code class="highlighter-rouge">git</code> is needed to later install <code class="highlighter-rouge">composer</code></li>
<li><code class="highlighter-rouge">libicu-dev</code> will be necessary for the internationalization php extension (<code class="highlighter-rouge">intl</code>)</li>
<li><code class="highlighter-rouge">libpq-dev</code> are the header files for postgres</li>
<li><code class="highlighter-rouge">libzip-dev</code> and <code class="highlighter-rouge">zlib1g-dev</code> are necessary for zip manipulation and needed to activate the corresponding php extension. Without them our <code class="highlighter-rouge">composer install</code> command will throw a lot of warnings and try to download each package twice <strong>slowing the whol process down for 2 or more minutes</strong></li>
</ul>
<p> </p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="pi">-</span> <span class="pi">></span>
<span class="no">docker-php-ext-install</span>
<span class="no">pdo_pgsql</span>
<span class="no">pgsql</span>
<span class="no">sockets</span>
<span class="no">intl</span>
<span class="no">zip</span></code></pre></figure>
<p><code class="highlighter-rouge">docker-php-ext-install</code> is a handy script for installing php extensions provided by the official php docker image. I restricted myself to the bare minimun of extension necessary for my project to run. You may need more.</p>
<p> </p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="pi">-</span> <span class="s">curl -sS https://getcomposer.org/installer | php</span>
<span class="pi">-</span> <span class="s">php composer.phar self-update</span>
<span class="pi">-</span> <span class="s">php composer.phar install --no-progress --no-interaction</span></code></pre></figure>
<p>Install <code class="highlighter-rouge">composer</code> and use it to install all of our project dependencies.</p>
<p> </p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="pi">-</span> <span class="s">cp .env.gitlab-ci .env</span></code></pre></figure>
<p>Set up the correct ENV variables. We’ll look into this file in a moment.</p>
<p> </p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="pi">-</span> <span class="s">php artisan key:generate</span>
<span class="pi">-</span> <span class="s">php artisan help config:clear</span>
<span class="pi">-</span> <span class="s">php artisan route:clear</span>
<span class="pi">-</span> <span class="s">php artisan migrate:refresh</span></code></pre></figure>
<p>Set application key, clear a couple of caches just in case and run the migrations.</p>
<p> </p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="s">test:app</span><span class="pi">:</span>
<span class="na">script</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">vendor/bin/phpunit --configuration phpunit-gitlabci.xml</span></code></pre></figure>
<p>And finally run our PHPUnit test suite using our <code class="highlighter-rouge">phpunit-gitlabci.xml</code> config file.</p>
<h2 id="small-break">Small break</h2>
<p>That was the most complicated file, the rest is a piece of cake, but you deserve a rest :P</p>
<iframe src="//giphy.com/embed/a3ANjL4bRwsO4" width="480" height="274" frameborder="0" class="giphy-embed" allowfullscreen=""></iframe>
<p> </p>
<h2 id="a-brief-look-into-envgitlab-ci">A brief look into <code class="highlighter-rouge">.env.gitlab-ci</code></h2>
<p>The file itself is quite self explainatory, and you can see the final version at the end of the article. But I wanted to highlight the database configuration.</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="s">DB_CONNECTION=pgsql</span>
<span class="s">DB_PORT=5432</span>
<span class="s">DB_HOST=postgres</span>
<span class="s">DB_DATABASE=mydb-test</span>
<span class="s">DB_USERNAME=runner</span>
<span class="s">DB_PASSWORD=</span></code></pre></figure>
<p>Notice how the values for <code class="highlighter-rouge">DB_HOST</code>, <code class="highlighter-rouge">DB_DATABASE</code> and <code class="highlighter-rouge">DB_USERNAME</code> are the same as those of inside of the <code class="highlighter-rouge">variables</code> key on the <code class="highlighter-rouge">.gitlab-ci.yml</code> configuration file.</p>
<p> </p>
<h2 id="a-brief-note-on-phpunit-gitlabcixml">A brief note on <code class="highlighter-rouge">phpunit-gitlabci.xml</code></h2>
<p>This file may be optional depending on your local configuration. If your normal <code class="highlighter-rouge">phpunit.xml</code> file doesn’t define any <code class="highlighter-rouge">DB_DATABASE</code> env variable you would not need this. But if it happens to define one, and has a different value from that that you chose for GitLab’s CI you have to remember to set it correctly:</p>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="s"><env name="DB_DATABASE" value="mydb-test"/></span></code></pre></figure>
<p> </p>
<h2 id="summary">Summary</h2>
<p>GitLab’s continuous integration system can be a blessing once is properly configured, but it can get a bit confusing if you trail off of the usual path and all you hear is <em>docker image configuration linking</em> and you’ve never worked with it before. Hopefully you’ll find this small guide useful
</p>
<div class="file-title">.gitlab-ci.yml</div>
<figure class="highlight"><pre><code class="language-yaml" data-lang="yaml"><span class="na">image</span><span class="pi">:</span> <span class="s">php:7.0</span>
<span class="na">services</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">postgres:latest</span>
<span class="na">variables</span><span class="pi">:</span>
<span class="na">POSTGRES_DB</span><span class="pi">:</span> <span class="s">mydb-test</span>
<span class="na">POSTGRES_USER</span><span class="pi">:</span> <span class="s">runner</span>
<span class="na">POSTGRES_PASSWORD</span><span class="pi">:</span> <span class="s2">"</span><span class="s">"</span>
<span class="na">before_script</span><span class="pi">:</span>
<span class="pi">-</span> <span class="pi">></span>
<span class="no">set -xe</span>
<span class="no">&& apt-get update -yqq</span>
<span class="no">&& apt-get install -yqq</span>
<span class="no">git</span>
<span class="no">libicu-dev</span>
<span class="no">libpq-dev</span>
<span class="no">libzip-dev</span>
<span class="no">zlib1g-dev</span>
<span class="pi">-</span> <span class="pi">></span>
<span class="no">docker-php-ext-install</span>
<span class="no">pdo_pgsql</span>
<span class="no">pgsql</span>
<span class="no">sockets</span>
<span class="no">intl</span>
<span class="no">zip</span>
<span class="pi">-</span> <span class="s">curl -sS https://getcomposer.org/installer | php</span>
<span class="pi">-</span> <span class="s">php composer.phar self-update</span>
<span class="pi">-</span> <span class="s">php composer.phar install --no-progress --no-interaction</span>
<span class="pi">-</span> <span class="s">cp .env.gitlab-ci .env</span>
<span class="pi">-</span> <span class="s">php artisan key:generate</span>
<span class="pi">-</span> <span class="s">php artisan help config:clear</span>
<span class="pi">-</span> <span class="s">php artisan route:clear</span>
<span class="pi">-</span> <span class="s">php artisan migrate:refresh</span>
<span class="s">test:app</span><span class="pi">:</span>
<span class="na">script</span><span class="pi">:</span>
<span class="pi">-</span> <span class="s">vendor/bin/phpunit --configuration phpunit-gitlabci.xml</span></code></pre></figure>
<p> </p>
<div class="file-title">.env.gitlab-ci</div>
<figure class="highlight"><pre><code class="language-sh" data-lang="sh"><span class="nv">APP_ENV</span><span class="o">=</span>testing
<span class="nv">APP_KEY</span><span class="o">=</span>key
<span class="nv">APP_DEBUG</span><span class="o">=</span><span class="nb">true
</span><span class="nv">APP_LOG_LEVEL</span><span class="o">=</span>debug
<span class="nv">APP_URL</span><span class="o">=</span>http://localhost:8000
<span class="nv">DB_CONNECTION</span><span class="o">=</span>pgsql
<span class="nv">DB_HOST</span><span class="o">=</span>postgres
<span class="nv">DB_PORT</span><span class="o">=</span>5432
<span class="nv">DB_DATABASE</span><span class="o">=</span>mydb-test
<span class="nv">DB_USERNAME</span><span class="o">=</span>runner
<span class="nv">DB_PASSWORD</span><span class="o">=</span>
<span class="nv">BROADCAST_DRIVER</span><span class="o">=</span>log
<span class="nv">CACHE_DRIVER</span><span class="o">=</span>array
<span class="nv">SESSION_DRIVER</span><span class="o">=</span>array
<span class="nv">QUEUE_DRIVER</span><span class="o">=</span>sync
<span class="nv">MAIL_DRIVER</span><span class="o">=</span>log</code></pre></figure>I recently set up a continuous integration process based on GitLab for a project based on Laravel and PostgreSQL. It took a while to get it right, so I am sharing here my final config.