pg_wal folder bloat problem on Standby nodes

Problem/Case
When using typical physical streaming, pg_wal size grows very fast and unexpectedly on DR (Disaster)/Backup/Standby node

Solution
Check inactive replication slots on DR node. If it exists, drop them

SQL> SELECT * FROM pg_replication_slots WHERE active = 'f';

Then run it:

SQL> SELECT pg_drop_replication_slot('inactive-node');

Problem/Case

When using patroni, pg_wal size grows very fast and unexpectedly on DR (Disaster)/Backup/Standby node

Solution

Check patroni maintenance mode by running "patronictl -c /etc/patroni/patroni.yml list"

If it is "pause" mode, then switch to "resume"

$ patronictl -c /etc/patroni/patroni.yml resume

psql cli tricks

-- show the connection info
# \conninfo

-- show IP address connected by the postgres client
# select inet_server_addr();

-- enable timing
# \timing
Timing is on.

-- enable expanded display
# \x
Expanded display is on.

-- run linux command inside psql command line
# \! date
Fri Sep  1 11:50:28 AM +03 2023

-- show the command history
# \s

-- prepare/set custom prompt:
# \set PROMPT1 '%M:%> user:%n database:%/ %l %x%# '
# \set PROMPT1 '%M:%> U:%n d:%/ %l %x%# '

PROMPT1 -> normal prompt
PROMPT2 -> if need to give more than one input command (extra prompt)
PROMPT3 -> if need to use "SQL COPY FROM STDIN" (extra prompt)

Ref: https://www.postgresql.org/docs/13/app-psql.html#APP-PSQL-PROMPTING

How to add a user into "wheel" group?

For example, you have only a root user for the super-user activities, but you wanted to use a custom user for some specific purposes. To make this, you might use "wheel" user group which is a special one for the "sudo" operations. 

Action

sudo usermod -aG wheel alperadm

Check and Determine the Option:
$ sudo cat /etc/sudoers

## Allows people in group wheel to run all commands
# %wheel  ALL=(ALL)       ALL

## Same thing without a password

%wheel        ALL=(ALL)       NOPASSWD: ALL

$ cat /etc/group | grep wheel
wheel:x:10:root,alperadm

Test
$ cat /etc/sudoers
cat: /etc/sudoers: Permission denied

$ sudo cat /etc/sudoers

Worked!

'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'stdout'

Action
$ ansible-playbook -i /etc/ansible/hosts /etc/ansible/example.yml -v

Error
'ansible.utils.unsafe_proxy.AnsibleUnsafeText object' has no attribute 'stdout'

Solution
Use the standard output of the variable before setting it as a fact

e.g.

- name: debug

  debug:

   msg: "Hello, {{ my_variable.stdout }}!"

- name: set fact the variable

  set_fact:

    fact_my_variable: "{{ my_variable.stdout }}"

"Attempting to decrypt but no vault secrets found" issue solution

Action

$ ansible-playbook -i /etc/ansible/hosts /etc/ansible/example.yml -v

Error
Attempting to decrypt but no vault secrets found

Solution
Add "--ask-vault-pass" OR "--vault-password-file <vault-file-path>" option to the end of the playbook command line

e.g.

$ ansible-playbook -i /etc/ansible/hosts /etc/ansible/example.yml -v --ask-vault-pass

el vs rhel

el (enterprise linux) e.g. centos, rocky linux, alma linux

rhel (red hat enterprise linux)

The main difference is that Red Hat provides technical support to the company/project if needs on RHEL. Also, there are various subscription options for being able to use RHEL for the companies at a cost.

A basic git exercise

e.g. an example with basic git commands 
source: develop branch 
target: private branch (not exist on the git local/remote repository at the begin)

1st step:
$ git clone http://blabla --branch develop
-- cloning develop branch data on the local machine

2nd step:
$ git branch private
-- defining a new branch as "private" on the local repository

3rd step:
$ git checkout private
-- switching the active/using branch to "private" on the local repository

4th step:
make your changes on the files/folders

5th step:
$ git status
-- see the summary that what you changed on the local repository

6th step:
$ git diff
-- see the differences between local repo and remote repo

7th step:
$ git add --all
-- add all modified files from (local) working directory to git staging area

8th step:
$ git commit -m "my latest changes applied on the code"
-- save your changes to the local repository to be ready for git push

9th step:
$ git push --set-upstream origin private
-- push all changes to the (remote) git repository by defining a new remote branch as the same local branch name; origin private

That's all!

Difficulty level: Low

How to resolve "[Errno 13] Permission denied"?

Action

$ ansible-playbook -i /etc/ansible/hosts /etc/ansible/example.yml -v

Error
ERROR! Unexpected Exception, this is probably a bug: [Errno 13] Permission denied

Analysis

ansible user does not have the owner permissions on "/dev/shm"

Possible Reason

Running "umount -a" or "umount /dev/shm"

Solution
$ sudo chown $USER /dev/shm

unzip vs gunzip

e.g. using unzip and gunzip tools for extracting the zip files

* example.txt file compressed into example.zip file

-> Listing contents of the zip file
$ unzip -l example.zip
$ gunzip -l example.zip

-> Extracting the zip file
$ unzip example.zip
$ gunzip -S .zip -c example.zip > example.txt

-> Extracting the zip file to specified directory
$ unzip example.zip -d /home/alper
$ gunzip -S .zip -c example.zip > /home/alper/example.txt

How to reset mariadb root password on using Galera Cluster system?

e.g. reset root password
two machines in a cluster

1st step:
-> Activate "safe_to_bootstrap" parameter on first machine
# vi /var/lib/mysql/grastate.dat
safe_to_bootstrap: 1

2nd step on first/second machine:
# systemctl stop mariadb

3rd step:
-> Run database in safe mode on first machine
# /usr/bin/mysqld_safe --skip-grant-tables --wsrep-new-cluster "${@:2}" > /dev/null 2>&1 & 

4th step:
-> Note the process id of mysql service
# ps -ef | grep mysql

5th step:
-> Connect mysql terminal then run following commands on first machine
# mysql -u root
 use mysql;
 update user SET PASSWORD=PASSWORD("password") WHERE USER='root';
 flush privileges;
 exit

6th step:
-> Try to connect with new password on first machine
# mysql -u root -ppassword

7th step:
-> kill the odd service on first machine
# kill -9 1111

8th step:
-> Start new cluster on first machine
# galera_new_cluster

9th step:
-> Start mariadb on second machine
$ systemctl start mariadb

Done...

Ref:
https://robbinespu.github.io/eng/2018/03/29/Reset_mariadb_root_password.html
https://blog.toonormal.com/2015/11/23/mariadb-cluster-install-notes/

e.g. preparing multiple virtual machines in local network

Hostnames: VM node-A, VM node-B, VM node-C

Adapter 1: Host-only Adapter (VirtualBox-Host-Only Ethernet Adapter)
Type -> Paravirtualized network adapter (virtio-net) - Deny Mode

Adapter 2: Bridged Adapter (Intel Ethernet Connection)
Type -> Paravirtualized network adapter (virtio-net) - Deny Mode

Test
Try to connect one of VMs such as "node-A" by SSH

How to solve git merge conflict?

e.g. git merge conflict issue
source: private branch
target: develop branch

1st step:
$ git clone http://blabla --branch private

2nd step:
$ cd <working folder>

3rd step:
$ git checkout private
-> if not specified branch by default while cloning branch

4th step:
$ git merge develop

Error occurred in here
Auto-merging my_new_file.yml
CONFLICT (content): Merge conflict in my_new_file.yml
Auto-merging my_new_file.yml
Automatic merge failed; fix conflicts and then commit the result.

5th step:
-> make necessary changes on my_new_file.yml

6th step:
$ git status
-> check the file

7th step:
$ git branch
-> check the branch before commit

8th step:
$ git add --all

9th step:
$ git commit -m "blabla"

10th step:
$ git push

Ansible Commands Book

-> valid syntax of the playbook
$ ansible-playbook -i myhosts example.yml -v --syntax-check

-> show elapsed time of the playbook
$ time ansible-playbook -i myhosts example.yml -v

-> maintain output verbosity (-v, -vv, -vvv, -vvv)
$ time ansible-playbook -i myhosts example.yml -vvvv

-> list tags/tasks of the playbook
$ ansible-playbook -i myhosts example.yml --list-tags
$ ansible-playbook -i myhosts example.yml --list-tasks

-> run the playbook by getting vault password/sudo password from user
$ ansible-playbook -i myhosts example.yml --ask-vault-pass
$ ansible-playbook -i myhosts example.yml --ask-sudo-pass

-> run the playbook with sudo password 
$ ansible-playbook -i myhosts example.yml -e "ansible_sudo_pass=alper"
(-e, --extra-vars)

How to drop database/user on logical replicated database?

1st step
on 2nd node:
$ echo "DROP SUBSCRIPTION exampledb_node_2" | psql

on 1st node:
$ echo "DROP PUBLICATION exampledb_node_1" | psql

2nd step
on 1st node:
$ echo -e "DROP DATABASE exampledb \n" | psql
$ echo "DROP USER \"user-example\"" | psql

on 2nd node:
$ echo -e "DROP DATABASE exampledb \n" | psql
$ echo "DROP USER \"user-example\"" | psql

Kubernetes label usage

e.g. namespace; kube-system

-> List labels

# kubectl get pods --all-namespaces --show-labels

# kubectl get pods -n kube-system --show-labels

-> List pods via labels
# kubectl get pods -n kube-system -l tier=control-plane

-> Describe pod/pods via labels
# kubectl describe pod -n kube-system -l component=kube-scheduler
# kubectl describe pod -n kube-system -l tier=control-plane

-> Get logs via labels

# kubectl logs -n kube-system -l component=kube-scheduler

# kubectl logs -n kube-system -l tier=control-plane

-> Delete pods via labels

# kubectl delete pods -n kube-system -l component=example

# kubectl delete pods -n kube-system -l 'component in (example,example2)'

# kubectl delete pods -n kube-system -l 'component notin (example3, example4)'

Examples of "escape sequences" usage on echo command

To use interpretation of backslash escapes, just need to put "-e" parameters afterward echo command.

Info: "-e" means enable escape sequences

Examples of backslash escapes:
New line;
$ echo -e "Hello World\n"
-> Hello World

Form feed;
$ echo -e "Hello World\fWhat a lovely day\fI like this\f"
-> Hello World
            What a lovely day
                              I like this

Carriage return;
$ echo -e "Hello World\rWorld"
-> World World

Horizontal tab;
$ echo -e "Hello World\tWhat a lovely day"
-> Hello World     What a lovely day

Vertical tab;
$ echo -e "Hello World\vWhat a lovely day"
-> Hello World
           What a lovely day

Null;
$ echo -e "Hello World\0 What a lovely day"
-> Hello World What a lovely day

Clear after all;
$ echo -e "Hello World\c What a lovely day"
-> Hello World

Escape character;

$ echo -e "Go to \\home directory"

-> Go to \home directory

$ echo -e "Go to \"home\" directory"

-> Go to "home" directory

Unicode usage;
$ echo -e "This is turkish lira symbol-> \u20ba"
-> This is turkish lira symbol-> ₺

Linux Terminal Commands Book

* chage (manage user password settings)
* hostnamectl (show hostname settings)
* date (manage system date and time)
* rpm (means RedHat Package Manager)
* timedatectl (show timezone settings)
* tcpdump (analyze network packages)
* tar (compress the files for archiving)
* tree (see the files as tree format)
* wc (means word count)
* yum (manage RPM packages on RedHat distro)

How to fix "connection refused" error?

Error
e.g.
The connection to the server 10.10.10.10:6443 was refused - did you specify the right host or port?

Solution
On kubernetes control nodes, try to "stop/start" docker service
# service docker restart
or
# systemctl restart docker

What are the differences of su/ su -/ sudo / sudo su in linux system?

Info: 
su means, switch user or substitute user
sudo means, super user do
to use sudo command, related user must be defined in sudoers file

$ su -
(switch user to root then executes all /etc/profile, .profile and .bashrc files by root)

$ su
(switch user to root user then executes only .bashrc file by root)

$ sudo su -
(switch user to root then executes all /etc/profile, .profile and .bashrc files by root if current user defined in sudoers file)

$ sudo su
(switch user to root then executes only .bashrc file by root if current user defined in sudoers file)

Ref: https://ozsoyler.blogspot.com/2016/09/how-to-gain-root-access-without_15.html

How to change timezone?

-> Determine your timezone
# cd /usr/share/zoneinfo
# tzselect
e.g.
Europe/Istanbul

-> Remove current localtime soft link
# rm /etc/localtime 

-> Define your localtime soft link as new
# ln -s /usr/share/zoneinfo/Europe/Istanbul /etc/localtime

-> Check your latest timezone setting
# timedatectl