public inbox for isar-users@googlegroups.com
 help / color / mirror / Atom feed
From: Uladzimir Bely <ubely@ilbers.de>
To: isar-users@googlegroups.com
Subject: [PATCH 1/3] testsuite: Dockerfile for isar-docker image
Date: Tue,  7 Mar 2023 05:36:47 +0100	[thread overview]
Message-ID: <20230307043649.12796-2-ubely@ilbers.de> (raw)
In-Reply-To: <20230307043649.12796-1-ubely@ilbers.de>

Add dockerfile and related files for generating isar-docker image
supposed to be used in docker-based CI systems.

Signed-off-by: Uladzimir Bely <ubely@ilbers.de>
---
 testsuite/dockerdata/.dockerignore        |   2 +
 testsuite/dockerdata/Dockerfile           |  48 ++++++
 testsuite/dockerdata/container-entrypoint |   9 ++
 testsuite/dockerdata/contrib/oe-git-proxy | 187 ++++++++++++++++++++++
 4 files changed, 246 insertions(+)
 create mode 100644 testsuite/dockerdata/.dockerignore
 create mode 100644 testsuite/dockerdata/Dockerfile
 create mode 100755 testsuite/dockerdata/container-entrypoint
 create mode 100755 testsuite/dockerdata/contrib/oe-git-proxy

diff --git a/testsuite/dockerdata/.dockerignore b/testsuite/dockerdata/.dockerignore
new file mode 100644
index 00000000..6e19512a
--- /dev/null
+++ b/testsuite/dockerdata/.dockerignore
@@ -0,0 +1,2 @@
+.dockerignore
+Dockerfile
diff --git a/testsuite/dockerdata/Dockerfile b/testsuite/dockerdata/Dockerfile
new file mode 100644
index 00000000..7372c752
--- /dev/null
+++ b/testsuite/dockerdata/Dockerfile
@@ -0,0 +1,48 @@
+FROM debian:bullseye-slim as isar-docker
+
+ARG TARGETPLATFORM
+ARG DEBIAN_FRONTEND=noninteractive
+
+RUN apt-get update && \
+    apt-get install -y locales && \
+    localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8
+
+ENV LANG=en_US.utf8
+ENV LC_ALL=en_US.UTF-8
+
+# Isar main dependencies
+RUN apt-get install -y -f --no-install-recommends \
+        binfmt-support bzip2 debootstrap dosfstools dpkg-dev gettext-base \
+        git mtools parted python3 quilt qemu-user-static reprepro sudo \
+        unzip xz-utils git-buildpackage pristine-tar sbuild schroot zstd \
+        python3-distutils \
+        umoci skopeo
+
+# Isar testsuite dependencies
+RUN apt-get install --no-install-recommends -y \
+        python3-pip && \
+    pip3 --proxy=$https_proxy install avocado-framework==100.1 && \
+    rm -rf $(pip3 cache dir) && \
+    apt-get install -y -f --no-install-recommends \
+        qemu-system ovmf
+
+RUN apt-get clean && \
+    rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* 
+
+COPY contrib/oe-git-proxy /usr/bin/
+ENV GIT_PROXY_COMMAND="oe-git-proxy" \
+    NO_PROXY="*"
+
+RUN echo "builder ALL=NOPASSWD: ALL" > /etc/sudoers.d/builder-nopasswd && \
+    chmod 660 /etc/sudoers.d/builder-nopasswd && \
+    echo "Defaults env_keep += \"ftp_proxy http_proxy https_proxy no_proxy\"" \
+    > /etc/sudoers.d/env_keep && chmod 660 /etc/sudoers.d/env_keep
+
+RUN useradd builder --user-group --create-home --home-dir /builder && \
+    sbuild-adduser builder >/dev/null 2>/dev/null
+
+COPY container-entrypoint /
+
+USER builder
+
+ENTRYPOINT ["/container-entrypoint"]
diff --git a/testsuite/dockerdata/container-entrypoint b/testsuite/dockerdata/container-entrypoint
new file mode 100755
index 00000000..6af07932
--- /dev/null
+++ b/testsuite/dockerdata/container-entrypoint
@@ -0,0 +1,9 @@
+#!/bin/bash
+
+sudo update-binfmts --enable && [ -f /proc/sys/fs/binfmt_misc/status ]
+
+if [ -n "$1" ]; then
+    exec "$@"
+else
+    exec bash
+fi
diff --git a/testsuite/dockerdata/contrib/oe-git-proxy b/testsuite/dockerdata/contrib/oe-git-proxy
new file mode 100755
index 00000000..aa9b9dc9
--- /dev/null
+++ b/testsuite/dockerdata/contrib/oe-git-proxy
@@ -0,0 +1,187 @@
+#!/bin/bash
+
+# oe-git-proxy is a simple tool to be via GIT_PROXY_COMMAND. It uses socat
+# to make SOCKS5 or HTTPS proxy connections.
+# It uses ALL_PROXY or all_proxy or http_proxy to determine the proxy server,
+# protocol, and port.
+# It uses NO_PROXY to skip using the proxy for a comma delimited list of
+# hosts, host globs (*.example.com), IPs, or CIDR masks (192.168.1.0/24). It
+# is known to work with both bash and dash shells.
+#
+# Example ALL_PROXY values:
+# ALL_PROXY=socks://socks.example.com:1080
+# ALL_PROXY=https://proxy.example.com:8080
+#
+# Copyright (c) 2013, Intel Corporation.
+#
+# SPDX-License-Identifier: GPL-2.0-only
+#
+# AUTHORS
+# Darren Hart <dvhart@linux.intel.com>
+
+# disable pathname expansion, NO_PROXY fields could start with "*" or be it
+set -f
+
+if [ $# -lt 2 -o "$1" = '--help' -o "$1" = '-h' ] ; then
+    echo 'oe-git-proxy: error: the following arguments are required: host port'
+    echo 'Usage: oe-git-proxy host port'
+    echo ''
+    echo 'OpenEmbedded git-proxy - a simple tool to be used via GIT_PROXY_COMMAND.'
+    echo 'It uses socat to make SOCKS or HTTPS proxy connections.'
+    echo 'It uses ALL_PROXY to determine the proxy server, protocol, and port.'
+    echo 'It uses NO_PROXY to skip using the proxy for a comma delimited list'
+    echo 'of hosts, host globs (*.example.com), IPs, or CIDR masks (192.168.1.0/24).'
+    echo 'It is known to work with both bash and dash shells.runs native tools'
+    echo ''
+    echo 'arguments:'
+    echo '  host                proxy host to use'
+    echo '  port                proxy port to use'
+    echo ''
+    echo 'options:'
+    echo '  -h, --help          show this help message and exit'
+    echo ''
+    exit 2
+fi
+
+# Locate the netcat binary
+if [ -z "$SOCAT" ]; then
+	SOCAT=$(which socat 2>/dev/null)
+	if [ $? -ne 0 ]; then
+		echo "ERROR: socat binary not in PATH" 1>&2
+		exit 1
+	fi
+fi
+METHOD=""
+
+# Test for a valid IPV4 quad with optional bitmask
+valid_ipv4() {
+	echo $1 | egrep -q "^([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(\.([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])){3}(/(3[0-2]|[1-2]?[0-9]))?$"
+	return $?
+}
+
+# Convert an IPV4 address into a 32bit integer
+ipv4_val() {
+	IP="$1"
+	SHIFT=24
+	VAL=0
+	for B in $( echo "$IP" | tr '.' ' ' ); do
+		VAL=$(($VAL+$(($B<<$SHIFT))))
+		SHIFT=$(($SHIFT-8))
+	done
+	echo "$VAL"
+}
+
+# Determine if two IPs are equivalent, or if the CIDR contains the IP
+match_ipv4() {
+	CIDR=$1
+	IP=$2
+
+	if [ -z "${IP%%$CIDR}" ]; then
+		return 0
+	fi
+
+	# Determine the mask bitlength
+	BITS=${CIDR##*/}
+	[ "$BITS" != "$CIDR" ] || BITS=32
+	if [ -z "$BITS" ]; then
+		return 1
+	fi
+
+	IPVAL=$(ipv4_val $IP)
+	IP2VAL=$(ipv4_val ${CIDR%%/*})
+
+	# OR in the unmasked bits
+	for i in $(seq 0 $((32-$BITS))); do
+		IP2VAL=$(($IP2VAL|$((1<<$i))))
+		IPVAL=$(($IPVAL|$((1<<$i))))
+	done
+
+	if [ $IPVAL -eq $IP2VAL ]; then
+		return 0
+	fi
+	return 1
+}
+
+# Test to see if GLOB matches HOST
+match_host() {
+	HOST=$1
+	GLOB=$2
+
+	if [ -z "${HOST%%*$GLOB}" ]; then
+		return 0
+	fi
+
+	# Match by netmask
+	if valid_ipv4 $GLOB; then
+		for HOST_IP in $(getent ahostsv4 $HOST | grep ' STREAM ' | cut -d ' ' -f 1) ; do
+			if valid_ipv4 $HOST_IP; then
+				match_ipv4 $GLOB $HOST_IP
+				if [ $? -eq 0 ]; then
+					return 0
+				fi
+			fi
+		done
+	fi
+
+	return 1
+}
+
+# If no proxy is set or needed, just connect directly
+METHOD="TCP:$1:$2"
+
+[ -z "${ALL_PROXY}" ] && ALL_PROXY=$all_proxy
+[ -z "${ALL_PROXY}" ] && ALL_PROXY=$http_proxy
+
+if [ -z "$ALL_PROXY" ]; then
+	exec $SOCAT STDIO $METHOD
+fi
+
+# Connect directly to hosts in NO_PROXY
+for H in $( echo "$NO_PROXY" | tr ',' ' ' ); do
+	if match_host $1 $H; then
+		exec $SOCAT STDIO $METHOD
+	fi
+done
+
+# Proxy is necessary, determine protocol, server, and port
+# extract protocol
+PROTO=${ALL_PROXY%://*}
+# strip protocol:// from string
+ALL_PROXY=${ALL_PROXY#*://}
+# extract host & port parts:
+#   1) drop username/password
+PROXY=${ALL_PROXY##*@}
+#   2) remove optional trailing /?
+PROXY=${PROXY%%/*}
+#   3) extract optional port
+PORT=${PROXY##*:}
+if [ "$PORT" = "$PROXY" ]; then
+	PORT=""
+fi
+#   4) remove port
+PROXY=${PROXY%%:*}
+
+# extract username & password
+PROXYAUTH="${ALL_PROXY%@*}"
+[ "$PROXYAUTH" = "$ALL_PROXY" ] && PROXYAUTH=
+[ -n "${PROXYAUTH}" ] && PROXYAUTH=",proxyauth=${PROXYAUTH}"
+
+if [ "$PROTO" = "socks" ] || [ "$PROTO" = "socks4a" ]; then
+	if [ -z "$PORT" ]; then
+		PORT="1080"
+	fi
+	METHOD="SOCKS4A:$PROXY:$1:$2,socksport=$PORT"
+elif [ "$PROTO" = "socks4" ]; then
+	if [ -z "$PORT" ]; then
+		PORT="1080"
+	fi
+	METHOD="SOCKS4:$PROXY:$1:$2,socksport=$PORT"
+else
+	# Assume PROXY (http, https, etc)
+	if [ -z "$PORT" ]; then
+		PORT="8080"
+	fi
+	METHOD="PROXY:$PROXY:$1:$2,proxyport=${PORT}${PROXYAUTH}"
+fi
+
+exec $SOCAT STDIO "$METHOD"
-- 
2.20.1


  reply	other threads:[~2023-03-07  4:36 UTC|newest]

Thread overview: 13+ messages / expand[flat|nested]  mbox.gz  Atom feed  top
2023-03-07  4:36 [PATCH 0/3] Switch to own " Uladzimir Bely
2023-03-07  4:36 ` Uladzimir Bely [this message]
2023-03-09  7:57   ` [PATCH 1/3] testsuite: Dockerfile for " Schaffner, Tobias
2023-03-09  9:05     ` Baurzhan Ismagulov
2023-03-09  9:14       ` Uladzimir Bely
2023-03-09 11:10         ` Jan Kiszka
2023-03-09 11:07       ` Jan Kiszka
2023-03-07  4:36 ` [PATCH 2/3] testsuite: Add README for isar-docker Uladzimir Bely
2023-03-07  4:36 ` [PATCH 3/3] gitlab-ci: Switch to own isar-docker image Uladzimir Bely
2023-03-14 13:49 ` [PATCH 0/3] " Henning Schild
2023-03-14 14:00   ` Jan Kiszka
2023-03-14 14:25     ` Henning Schild
2023-03-14 14:28       ` Jan Kiszka

Reply instructions:

You may reply publicly to this message via plain-text email
using any one of the following methods:

* Save the following mbox file, import it into your mail client,
  and reply-to-all from there: mbox

  Avoid top-posting and favor interleaved quoting:
  https://en.wikipedia.org/wiki/Posting_style#Interleaved_style

* Reply using the --to, --cc, and --in-reply-to
  switches of git-send-email(1):

  git send-email \
    --in-reply-to=20230307043649.12796-2-ubely@ilbers.de \
    --to=ubely@ilbers.de \
    --cc=isar-users@googlegroups.com \
    /path/to/YOUR_REPLY

  https://kernel.org/pub/software/scm/git/docs/git-send-email.html

* If your mail client supports setting the In-Reply-To header
  via mailto: links, try the mailto: link
Be sure your reply has a Subject: header at the top and a blank line before the message body.
This is a public inbox, see mirroring instructions
for how to clone and mirror all data and code used for this inbox