From mboxrd@z Thu Jan 1 00:00:00 1970 X-GM-THRID: 7207658688501252096 X-Received: by 2002:a05:600c:1:b0:3df:d8c9:caa5 with SMTP id g1-20020a05600c000100b003dfd8c9caa5mr2766562wmc.1.1678163811857; Mon, 06 Mar 2023 20:36:51 -0800 (PST) X-BeenThere: isar-users@googlegroups.com Received: by 2002:a05:600c:1ca4:b0:3e2:165f:33d0 with SMTP id k36-20020a05600c1ca400b003e2165f33d0ls6342063wms.2.-pod-control-gmail; Mon, 06 Mar 2023 20:36:50 -0800 (PST) X-Google-Smtp-Source: AK7set+QeD2dfQwsnBP9oxU9zcygkox9dLE1E0vc1NYKn0LS5tQlyY+lfkdU/g5rjNVbVlL8UKTX X-Received: by 2002:a05:600c:1f06:b0:3ea:f73e:9d8d with SMTP id bd6-20020a05600c1f0600b003eaf73e9d8dmr12327544wmb.32.1678163810300; Mon, 06 Mar 2023 20:36:50 -0800 (PST) ARC-Seal: i=1; a=rsa-sha256; t=1678163810; cv=none; d=google.com; s=arc-20160816; b=TaDOBYXj3FGZTQOuQKfi7Uf/Sw8TqHnR1jRZk3PHSW0KYams/nJt3NZQS2gxR58usk 34dA51Uy/EIhHdDbq6HZiHB0N3DX7Y+04/DzhlOGol1KwZyTRtcnMI24Nk5IQENE1VDw O76rVzQcHWnEXaXgcB4xCvFcWLzNKROoBKEz5e61MjXHjgoMb+A/BleXHqB/U3aw0h31 bmLWkTMpnVbTEiFMmfkHaY9UXbT6ARCvGEp88YFaVkE7yuYsZK7J2YABrLcnrAtoTXBK Y2G43j4QPIKjySr6tH7rDFcYX683ShmLoi6w9d1n7F4HEsLWSknoyvA6I+jF8ZiZYZep +HtA== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20160816; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:to:from; bh=NCJ1znb2sZkxmCWRZRsUBK7lpLR6oXUTPiEEebak6cs=; b=EH2/QDJirz2NP3ea2dgKhLNzknldKa6TFxNC31W41k/Q9zgJpyPGrepyPGn5K92hEx cnlaR6qkYT1O2gwcMxki0cMTJ8x5+T/yE4yoK5W/2DJz8nllNRGZLkWmtCrMw+7uOP63 e6dgbNDAXn/U3dhcAwbzBdLoNsTVWETsbUiuv0CC6o04cJfbW/MACA0xYEOt+AqofD0I +IY6oLcHwTtfJYjTQ3q3mKCf9UPbTsI5bmLQxktzzYI6hCBG9XOIhy7vJvGmqZXVHv6q wANdPGD/ojgCFGxYjK5PUNTG5+9ZcJllGYATRAicx4/kB1fOcJuHGe3UmC/ZdRgX2r2O su9g== ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Return-Path: Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id p26-20020a05600c1d9a00b003dc537184cfsi18812wms.1.2023.03.06.20.36.50 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Mon, 06 Mar 2023 20:36:50 -0800 (PST) Received-SPF: pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of ubely@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=ubely@ilbers.de Received: from baighyz.m.ilbers.de (host-80-81-17-52.static.customer.m-online.net [80.81.17.52]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 3274anLu018531 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Tue, 7 Mar 2023 05:36:49 +0100 From: Uladzimir Bely To: isar-users@googlegroups.com Subject: [PATCH 1/3] testsuite: Dockerfile for isar-docker image Date: Tue, 7 Mar 2023 05:36:47 +0100 Message-Id: <20230307043649.12796-2-ubely@ilbers.de> X-Mailer: git-send-email 2.20.1 In-Reply-To: <20230307043649.12796-1-ubely@ilbers.de> References: <20230307043649.12796-1-ubely@ilbers.de> MIME-Version: 1.0 Content-Transfer-Encoding: 8bit X-Spam-Status: No, score=-1.0 required=5.0 tests=ALL_TRUSTED autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-TUID: 29nNzvisR3Xn Add dockerfile and related files for generating isar-docker image supposed to be used in docker-based CI systems. Signed-off-by: Uladzimir Bely --- testsuite/dockerdata/.dockerignore | 2 + testsuite/dockerdata/Dockerfile | 48 ++++++ testsuite/dockerdata/container-entrypoint | 9 ++ testsuite/dockerdata/contrib/oe-git-proxy | 187 ++++++++++++++++++++++ 4 files changed, 246 insertions(+) create mode 100644 testsuite/dockerdata/.dockerignore create mode 100644 testsuite/dockerdata/Dockerfile create mode 100755 testsuite/dockerdata/container-entrypoint create mode 100755 testsuite/dockerdata/contrib/oe-git-proxy diff --git a/testsuite/dockerdata/.dockerignore b/testsuite/dockerdata/.dockerignore new file mode 100644 index 00000000..6e19512a --- /dev/null +++ b/testsuite/dockerdata/.dockerignore @@ -0,0 +1,2 @@ +.dockerignore +Dockerfile diff --git a/testsuite/dockerdata/Dockerfile b/testsuite/dockerdata/Dockerfile new file mode 100644 index 00000000..7372c752 --- /dev/null +++ b/testsuite/dockerdata/Dockerfile @@ -0,0 +1,48 @@ +FROM debian:bullseye-slim as isar-docker + +ARG TARGETPLATFORM +ARG DEBIAN_FRONTEND=noninteractive + +RUN apt-get update && \ + apt-get install -y locales && \ + localedef -i en_US -c -f UTF-8 -A /usr/share/locale/locale.alias en_US.UTF-8 + +ENV LANG=en_US.utf8 +ENV LC_ALL=en_US.UTF-8 + +# Isar main dependencies +RUN apt-get install -y -f --no-install-recommends \ + binfmt-support bzip2 debootstrap dosfstools dpkg-dev gettext-base \ + git mtools parted python3 quilt qemu-user-static reprepro sudo \ + unzip xz-utils git-buildpackage pristine-tar sbuild schroot zstd \ + python3-distutils \ + umoci skopeo + +# Isar testsuite dependencies +RUN apt-get install --no-install-recommends -y \ + python3-pip && \ + pip3 --proxy=$https_proxy install avocado-framework==100.1 && \ + rm -rf $(pip3 cache dir) && \ + apt-get install -y -f --no-install-recommends \ + qemu-system ovmf + +RUN apt-get clean && \ + rm -rf /var/lib/apt/lists/* /tmp/* /var/tmp/* + +COPY contrib/oe-git-proxy /usr/bin/ +ENV GIT_PROXY_COMMAND="oe-git-proxy" \ + NO_PROXY="*" + +RUN echo "builder ALL=NOPASSWD: ALL" > /etc/sudoers.d/builder-nopasswd && \ + chmod 660 /etc/sudoers.d/builder-nopasswd && \ + echo "Defaults env_keep += \"ftp_proxy http_proxy https_proxy no_proxy\"" \ + > /etc/sudoers.d/env_keep && chmod 660 /etc/sudoers.d/env_keep + +RUN useradd builder --user-group --create-home --home-dir /builder && \ + sbuild-adduser builder >/dev/null 2>/dev/null + +COPY container-entrypoint / + +USER builder + +ENTRYPOINT ["/container-entrypoint"] diff --git a/testsuite/dockerdata/container-entrypoint b/testsuite/dockerdata/container-entrypoint new file mode 100755 index 00000000..6af07932 --- /dev/null +++ b/testsuite/dockerdata/container-entrypoint @@ -0,0 +1,9 @@ +#!/bin/bash + +sudo update-binfmts --enable && [ -f /proc/sys/fs/binfmt_misc/status ] + +if [ -n "$1" ]; then + exec "$@" +else + exec bash +fi diff --git a/testsuite/dockerdata/contrib/oe-git-proxy b/testsuite/dockerdata/contrib/oe-git-proxy new file mode 100755 index 00000000..aa9b9dc9 --- /dev/null +++ b/testsuite/dockerdata/contrib/oe-git-proxy @@ -0,0 +1,187 @@ +#!/bin/bash + +# oe-git-proxy is a simple tool to be via GIT_PROXY_COMMAND. It uses socat +# to make SOCKS5 or HTTPS proxy connections. +# It uses ALL_PROXY or all_proxy or http_proxy to determine the proxy server, +# protocol, and port. +# It uses NO_PROXY to skip using the proxy for a comma delimited list of +# hosts, host globs (*.example.com), IPs, or CIDR masks (192.168.1.0/24). It +# is known to work with both bash and dash shells. +# +# Example ALL_PROXY values: +# ALL_PROXY=socks://socks.example.com:1080 +# ALL_PROXY=https://proxy.example.com:8080 +# +# Copyright (c) 2013, Intel Corporation. +# +# SPDX-License-Identifier: GPL-2.0-only +# +# AUTHORS +# Darren Hart + +# disable pathname expansion, NO_PROXY fields could start with "*" or be it +set -f + +if [ $# -lt 2 -o "$1" = '--help' -o "$1" = '-h' ] ; then + echo 'oe-git-proxy: error: the following arguments are required: host port' + echo 'Usage: oe-git-proxy host port' + echo '' + echo 'OpenEmbedded git-proxy - a simple tool to be used via GIT_PROXY_COMMAND.' + echo 'It uses socat to make SOCKS or HTTPS proxy connections.' + echo 'It uses ALL_PROXY to determine the proxy server, protocol, and port.' + echo 'It uses NO_PROXY to skip using the proxy for a comma delimited list' + echo 'of hosts, host globs (*.example.com), IPs, or CIDR masks (192.168.1.0/24).' + echo 'It is known to work with both bash and dash shells.runs native tools' + echo '' + echo 'arguments:' + echo ' host proxy host to use' + echo ' port proxy port to use' + echo '' + echo 'options:' + echo ' -h, --help show this help message and exit' + echo '' + exit 2 +fi + +# Locate the netcat binary +if [ -z "$SOCAT" ]; then + SOCAT=$(which socat 2>/dev/null) + if [ $? -ne 0 ]; then + echo "ERROR: socat binary not in PATH" 1>&2 + exit 1 + fi +fi +METHOD="" + +# Test for a valid IPV4 quad with optional bitmask +valid_ipv4() { + echo $1 | egrep -q "^([1-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])(\.([0-9]|[1-9][0-9]|1[0-9][0-9]|2[0-4][0-9]|25[0-5])){3}(/(3[0-2]|[1-2]?[0-9]))?$" + return $? +} + +# Convert an IPV4 address into a 32bit integer +ipv4_val() { + IP="$1" + SHIFT=24 + VAL=0 + for B in $( echo "$IP" | tr '.' ' ' ); do + VAL=$(($VAL+$(($B<<$SHIFT)))) + SHIFT=$(($SHIFT-8)) + done + echo "$VAL" +} + +# Determine if two IPs are equivalent, or if the CIDR contains the IP +match_ipv4() { + CIDR=$1 + IP=$2 + + if [ -z "${IP%%$CIDR}" ]; then + return 0 + fi + + # Determine the mask bitlength + BITS=${CIDR##*/} + [ "$BITS" != "$CIDR" ] || BITS=32 + if [ -z "$BITS" ]; then + return 1 + fi + + IPVAL=$(ipv4_val $IP) + IP2VAL=$(ipv4_val ${CIDR%%/*}) + + # OR in the unmasked bits + for i in $(seq 0 $((32-$BITS))); do + IP2VAL=$(($IP2VAL|$((1<<$i)))) + IPVAL=$(($IPVAL|$((1<<$i)))) + done + + if [ $IPVAL -eq $IP2VAL ]; then + return 0 + fi + return 1 +} + +# Test to see if GLOB matches HOST +match_host() { + HOST=$1 + GLOB=$2 + + if [ -z "${HOST%%*$GLOB}" ]; then + return 0 + fi + + # Match by netmask + if valid_ipv4 $GLOB; then + for HOST_IP in $(getent ahostsv4 $HOST | grep ' STREAM ' | cut -d ' ' -f 1) ; do + if valid_ipv4 $HOST_IP; then + match_ipv4 $GLOB $HOST_IP + if [ $? -eq 0 ]; then + return 0 + fi + fi + done + fi + + return 1 +} + +# If no proxy is set or needed, just connect directly +METHOD="TCP:$1:$2" + +[ -z "${ALL_PROXY}" ] && ALL_PROXY=$all_proxy +[ -z "${ALL_PROXY}" ] && ALL_PROXY=$http_proxy + +if [ -z "$ALL_PROXY" ]; then + exec $SOCAT STDIO $METHOD +fi + +# Connect directly to hosts in NO_PROXY +for H in $( echo "$NO_PROXY" | tr ',' ' ' ); do + if match_host $1 $H; then + exec $SOCAT STDIO $METHOD + fi +done + +# Proxy is necessary, determine protocol, server, and port +# extract protocol +PROTO=${ALL_PROXY%://*} +# strip protocol:// from string +ALL_PROXY=${ALL_PROXY#*://} +# extract host & port parts: +# 1) drop username/password +PROXY=${ALL_PROXY##*@} +# 2) remove optional trailing /? +PROXY=${PROXY%%/*} +# 3) extract optional port +PORT=${PROXY##*:} +if [ "$PORT" = "$PROXY" ]; then + PORT="" +fi +# 4) remove port +PROXY=${PROXY%%:*} + +# extract username & password +PROXYAUTH="${ALL_PROXY%@*}" +[ "$PROXYAUTH" = "$ALL_PROXY" ] && PROXYAUTH= +[ -n "${PROXYAUTH}" ] && PROXYAUTH=",proxyauth=${PROXYAUTH}" + +if [ "$PROTO" = "socks" ] || [ "$PROTO" = "socks4a" ]; then + if [ -z "$PORT" ]; then + PORT="1080" + fi + METHOD="SOCKS4A:$PROXY:$1:$2,socksport=$PORT" +elif [ "$PROTO" = "socks4" ]; then + if [ -z "$PORT" ]; then + PORT="1080" + fi + METHOD="SOCKS4:$PROXY:$1:$2,socksport=$PORT" +else + # Assume PROXY (http, https, etc) + if [ -z "$PORT" ]; then + PORT="8080" + fi + METHOD="PROXY:$PROXY:$1:$2,proxyport=${PORT}${PROXYAUTH}" +fi + +exec $SOCAT STDIO "$METHOD" -- 2.20.1