Find and Grep utilities
The powerful find and grep utilities on Unix/Linux : usage and practical examples. Beyond the basic cases, the examples address more complex needs with sometimes the power of regular expressions. A few examples covered : deleting files based on criteria, computing the size of images in a directory, finding invalid symbolic links. This paper is quite old (first version in 2002) but is enriched by the time with examples of interest to remember.
awk, nawk and gawk utilities - Tutorial
The awk, nawk and gawk (GNU Awk) utilities are very easy to use text file parsers. They handle efficiently text files of data delimited by a character. With an easy syntax to understand, operations for filtering rows, filtering columns, enriching content, converting formats, computing aggregates (averages, sums for example), etc. become child’s play with these utilities. awk, gawk and nawk differ only in a few very advanced features. awk is the utility to use without hesitation to parse very efficiently in seconds complex log files for example. Getting started with the nawk utility through examples, and contrary to common beliefs it is also available on Windows platforms.
Google PageSpeed - LightHouse 6, new features CLS (Cumulative Layout Shift) and LCP (Largest Contentful Paint)
Web pages performance metrics are modified in LightHouse 6, they are more realistic and close to the user experience. A very interesting metric is introduced: CLS (Cumulative Layout Shift). The CLS statistics quantify the visual stability of web pages to avoid bad user experiences. All the tools are available with Chrome and LightHouse to diagnose the elements concerned in the visual instability of a page and in a few steps, the CLS score can be optimized, but everything will of course depend on the complexity of the pages and code.
Mathematical expressions and equations in HTML with MathJax and AsciiMath
Measuring and storing pages performance metrics using Google PageSpeed Insights API and Python
Page speed is one of the key criteria in Google search engines ranking algorithms. The API Google PageSpeed Insights gives the ability to retrieve programmatically page speed LightHouse results executed on web pages from Google labs. With few Python code lines, the valuable statistics can be extracted and stored in a table for historization : useful when planning benchmarks before/after upgrades, updates.
Python, application configuration - environment variables, ini and YAML files
Obviously, no hard-coded application configuration in programs, Python programs or not. How to read (write) configuration data with Python from environment variables, INI files or YAML files? The os, configparser and PyYAML packages meet these needs. configparser for INI files and yaml for YAML files have their advantages and inconvenients if we venture to compare these 2 packages to make a choice. The JSON format is not discussed here, a dedicated article is published on this topic.
Python, handling dictionaries with dot notation
Python, Managing HTTP requests with the packages requests and httplib2
HTTP requests with the GET or POST method are widely used when exchanging data through programming API. With Python, 2 interesting packages for executing HTTP requests: requests and httplib2. The requests package is the easiest syntactically and natively includes a JSON decoder. About the httplib2 package, a very interesting feature: a caching mechanism to avoid unnecessary network calls in some application contexts using relatively static data.
Python, Parsing arguments with the packages argparse and getopt
2 native packages are available with Python to efficiently manage the command lines and arguments of a Python program: argparse and getopt. argparse is by far the most advanced package and offers the most productivity, about getopt a very strong resemblance to its habits when handling arguments in shell or in C language.
Python, Reading and writing JSON with the package json
When developing a Web site, event listeners are implemented, by third party libraries or by your own. In some circumstances, when starting performance improvements tasks or when debugging behavior issues in event listeners due to a third party library, we may need to get an overview of all event listeners. How to get the full list : the events defined with addEventListener and the ones defined with the corresponding attribute ? It's not so trivial.
Standalone MariaDB ColumnStore 1.2.3 installation on Ubuntu - Getting started
In order to discover the product, the installation of a standalone MariaDB ColumnStore system on Ubuntu is rather simple. Getting started is also quick with the combination of the mcsadmin and mcsmysql utilities. A technical starting point on this column-oriented database for real-time analytical processing of distributed and very high volume data.
How to list members of an Active directory group defined in Microsoft SQL Server
Project managers often directly ask the DBAs to provide the list of accounts belonging to a Microsoft Active Directory group defined in Microsoft SQL Server. Basic command lines exist (net group, net user) to list the members without having to disturb the administrators of the Active Directory domain controller (AD).
InfluxDB v2, Getting started. Preparing the migration from version 1.7
The migration to InfluxDB v2 will need adjustments, many InfluxDB v1 functionalities are replaced in version 2. Migration procedure from version 1.x to version 2 is not yet unveiled, surely it will preconize export/import methods. The terminology is a little bit modified in version 2 : a database is now a bucket and an organization is necessarily attached to a bucket. No changes about the InfluxDB Line protocol. In this article : getting started with InfluxDB v2 and the breaks to manage when scheduling migration from version 1.x InfluQL is replaced by the language Flux, Continuous queries are replaced by tasks. Native support of the protocols Graphite, OpenTSDB… is removed, Telegraf will have to be used. Grafana plugin for Flux is not yet fully ready.
Protecting SSH port with Fail2ban on Ubuntu - Setup and configuration
On Ubuntu servers, ufw (Uncomplicated Firewall) is a good tool to apply firewall rules on ports without having to use iptables commands which are quite difficult, but it's not enough to protect port 22 for SSH services. Fail2Ban is the complementary tool to ufw in order to protect an Ubuntu system from attacks on SSH. Fail2Ban is a Python package which interacts like ufw with iptables and can be installed in a Python virtual environment, this is the case in this article.
Understanding, installing and using the time series database InfluxDB 1.7
InfluxDB is a high performance time series database with efficient compression. The architecture is well designed (measurements retentions, shards) with easy ingestion, natively or through common time series protocols (OpenTSDB, Graphite…). Queries are SQL Like and the reporting is intuitive with Grafana or Chronograf. This introduction to InfluxDB v 1.7 also points out the breaks with the next major version 2.0.
Python - Understanding and demistifying virtualenv
With Python, when installing a product, many dependencies can also be installed. It becomes tedious to manage polluting the Python global distribution : dependencies, package versions conflicts, binaries… The packages used by the most users can be installed in the system distribution and packages needed to only one user/product in a virtual environment using virtualenv. How to create and use virtual environments with virtualenv ? How packages and versions are managed in virtual and system distributions ?
PostgreSQL - Planned Failover / Failback procedures with streaming replication
Many articles deal with promoting a PostgreSQL standby server to primary server using the command pg_ctl promote. Many of them conclude the standby server shoud be be rebuilt when primary server is back. What about a scheduled maintenance on the primary server including shutdowns ? Do we have to rebuild the primary/standby system ? Obviously the answer is : NO. Assuming there is enough space disk for WAL files retention during the maintenance task, if the replication slot and the recovery option recovery_timeline_target are cautiously managed during the failover/failback procedures, there is no need to rebuild the whole standby system implying backup/restore databases and/or to use pg_rewind.
Streaming replication with PostgreSQL 9.6, 10 and 11 - PostgreSQL Standby Databases
Implementing a PostgreSQL 9.6 streaming replication on Linux Ubuntu 18.04 servers is very easy. In the streaming replication mode, the standby connects to the primary, which streams WAL records (Write Ahead Log) to the standby as they're generated, without waiting for the WAL file to be filled. Streaming replication allows a standby server to stay more up-to-date than is possible with file-based log shipping. The standby server can be in read only mode for reporting purposes for example. A replication slot ensures that the needed WAL files for the standby are not removed in the primary server before the standby server processes them. Multiple replication slots can be defined depending on the number of the standby servers.
PostgreSQL 9.6 - Quick setup and installation on Ubuntu 18.04
A PostgreSQL 9.6 instance is installed and customized (port, database and configuration files…) on Ubuntu 18.04. A quick guide for creating a PostgreSQL database and managing this database with pg_ctl (start, stop, restart…)