From mboxrd@z Thu Jan 1 00:00:00 1970 Return-Path: Received: from shymkent.ilbers.de ([unix socket]) by shymkent (Cyrus 2.5.10-Debian-2.5.10-3+deb9u2) with LMTPA; Wed, 11 Mar 2026 12:15:46 +0100 X-Sieve: CMU Sieve 2.4 Received: from mail-ej1-f61.google.com (mail-ej1-f61.google.com [209.85.218.61]) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPS id 62BBFjKQ022646 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT) for ; Wed, 11 Mar 2026 12:15:45 +0100 Received: by mail-ej1-f61.google.com with SMTP id a640c23a62f3a-b8f5bce308dsf1226129866b.1 for ; Wed, 11 Mar 2026 04:15:45 -0700 (PDT) ARC-Seal: i=2; a=rsa-sha256; t=1773227740; cv=pass; d=google.com; s=arc-20240605; b=Qvn0PcmY+Y5cbCzauv2jcklEO0mVU+YfMT1UXf494m4QpX8ZXHWYSsIq8GRjdomUsm UFqIzIsBmt4AutTm7PFkaSkA1H/neAzkULGjHNdyObbtNpawH44B2FSyVHaL+lc7JcIp wbj7dRDBedFNUhCRU7IYKAjkQt37w4Mt6N6Z5/TUqHdcwB1gsjEfccc5A/o4FAW9Qu9k VPWdFK63sHfOF2OTsdqg3pp/H9tKssfA8blgSRW9rbKFUXL9DGGiCX5pjyaC7SH4IImq G48lZcvlnLi9bVhngU4MeyJ/DU9u810Th13L96VifsOFfBOH51btw12UDg9rJyLq/rj/ 7nrA== ARC-Message-Signature: i=2; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from:sender:dkim-signature; bh=0VKETf9KlRRKaOVsmIIX5Uu9OvD2ALc4ehNaj61FzDg=; fh=tey8hJfeUGEatdOia7id1OUrf+pWJIl+l+6l+x8KvRY=; b=RnroiW4kKM3TkBuosTVBBIHJnOSQl1fLsXnomuBnBhT85gSc1da0qJpMTpnBkUyRgV L9sUPRnwKK84XCvMs1hxS9sfyeqA6Q4AbMEupFF0W2dmWRiQGlpJl3vX8rVjt3AqhFNA x3AvRz++xcgS+I6T66C4YVjrWRfuj/y4ke0Qh26BDVRCrOjpv4yHzUtx9bMMgBYIMbcd 3KntGFep8EEvf4ml9CRgjEg4gsYnw17dbrKsG7klNJSRN00w6gZ4arZ0fnhiZVRlj+H8 u+RlMVLTimXEmhQYGgIDBlTJwLBQ7w4mmei2dJfde/pdMu5iDYY+raEeCv1JPS6rZcwC VQgA==; darn=ilbers.de ARC-Authentication-Results: i=2; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=googlegroups.com; s=20230601; t=1773227740; x=1773832540; darn=ilbers.de; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :list-id:mailing-list:precedence:x-original-authentication-results :x-original-sender:mime-version:references:in-reply-to:message-id :date:subject:cc:to:from:sender:from:to:cc:subject:date:message-id :reply-to; bh=0VKETf9KlRRKaOVsmIIX5Uu9OvD2ALc4ehNaj61FzDg=; b=vbTvHpcpKTdrkmqDXNF4tRMNNeGsDZK9pEcHlSOGnUzvOGlSaNVF93BZPD+kYqqYnv Poqqwxw3z9FCsVODyPrWZWTHEFM0uY0d8hmObvGWMGw6i913KGl9VVm8LPKqMyNwGh6G C4WZbLqnTsuSvsBVtq5xx2iS4/9/uzMAxPMxP3q5aSWoSLqG/+fPNlqOBcVxxJMflcM2 O4tGryLSGoTf05SivLDiRrvEAZj9oBhARTXaoj6fRCHbr0NUxKgh3vCIqZUR1f9bPcMm SMxA/JlCKgXBQSYcLtRibwGNrpYA5W0nQxFTCJpTvleXNnz9VwDt4moP/7FdpRn7Y8nD 4xXA== X-Google-DKIM-Signature: v=1; a=rsa-sha256; c=relaxed/relaxed; d=1e100.net; s=20230601; t=1773227740; x=1773832540; h=list-unsubscribe:list-subscribe:list-archive:list-help:list-post :x-spam-checked-in-group:list-id:mailing-list:precedence :x-original-authentication-results:x-original-sender:mime-version :references:in-reply-to:message-id:date:subject:cc:to:from :x-beenthere:x-gm-message-state:sender:from:to:cc:subject:date :message-id:reply-to; bh=0VKETf9KlRRKaOVsmIIX5Uu9OvD2ALc4ehNaj61FzDg=; b=L3kE8haMyPAoiu1OWS8rCtnGjKVLRRYSjNtHWVpuiNWIIZyAacGn5G+4q8SXyVeOgj zqIOJ/57UiZEbNgwhXPcG2P0FJ9ulSmJoZUaB/xmnIxfrR8hEo8nADHHwAUAAvzvq9Rr uLMtiJ5I7Gg7Z9oU+HNcxgSvekF/GENfK2kGDqcNq8bfS9F0tapzJ/CjFvCuefbhse6S 4p9YcApg+8H2N67FQPZBghOVPq40Gbwc5mAslqUnlggXf+cqEszeadfd2fOPfD4eo3/1 CDwsVy1iUzcObjYUdLmb5VG2OabPkPMphrX6kUxnmC/lGHSa5aPe4muEj+U4wuHI88nM rZiw== Sender: isar-users@googlegroups.com X-Forwarded-Encrypted: i=2; AJvYcCWZRB1Xeb/jXP+5T8MWuZGB4OkYdT49qdUNyS2zY3m1esNPuDSSJ/eqUnbJmRw3N7tomd3n@ilbers.de X-Gm-Message-State: AOJu0YwEGUNyAg7yZDyxriCV5SW+DjuWRrEwBhTywXZqIELJxJlqhzVp jk9iUI96XodwDrpwwE+zEgSRMne3avN/jVb0tJKAImPQNLU9G4mR82Oc X-Received: by 2002:a05:6402:1471:b0:661:39a2:f530 with SMTP id 4fb4d7f45d1cf-663196d11a0mr1198466a12.12.1773227739607; Wed, 11 Mar 2026 04:15:39 -0700 (PDT) X-BeenThere: isar-users@googlegroups.com; h="AV1CL+EglF+/im+F0quVuTeWTRD9eSZVL+M5dc2jM98H0hMWEA==" Received: by 2002:aa7:cccd:0:b0:658:2f63:8d83 with SMTP id 4fb4d7f45d1cf-6612ead24cbls2526101a12.0.-pod-prod-01-eu; Wed, 11 Mar 2026 04:15:37 -0700 (PDT) X-Received: by 2002:a05:6402:3785:b0:662:b0cf:b997 with SMTP id 4fb4d7f45d1cf-66319cd6b9bmr1079731a12.16.1773227737467; Wed, 11 Mar 2026 04:15:37 -0700 (PDT) ARC-Seal: i=1; a=rsa-sha256; t=1773227737; cv=none; d=google.com; s=arc-20240605; b=gDmC2zm7eXckASUAurb4uAhWZvFHEb1cmV6se0adeYpG31QuUkPVxim07M9FOnp2eT bFMQbTabRuw+g0UsDcMTo+J7D41asCtYu2+YMfaAuO99v5uTTdKCP/mrARBoMVH5NOTs CnzSdlXWz0m50YqC0QEZxCcb1YjsXKGHH+1xbnVG6HTsRCy7mBPBwk38ZdukpcUu8c/u DhugXXQhgTzRpx9eq8pPgjMsZePTH0XgDynsymZ6K1kLvbvC1HElmllEdAa4ooCRWm90 BqVTS5JTvDNhtH/L/PjcEpKKmNHLTXkJForeKxh3aLswMhM9aBrHOnxuAMrID7cEh49M 1gkg== ARC-Message-Signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=google.com; s=arc-20240605; h=content-transfer-encoding:mime-version:references:in-reply-to :message-id:date:subject:cc:to:from; bh=U2TH5qdsLgWZuOcrnxmlOAd1Idpsv1KEtU7/GECASpo=; fh=PHAUAmPq/1fgjgwS4i7Spq2ceipXP7+JEnvrX42Opjg=; b=j/yQ4QMtPUuN33Y+yexiu/93j1+UhZuKgGQLWaBKuHkzdjcMaIDD/qjxAuQdHzR3tK KRrQobetKrox5sdIiLo+qSqRYQwf7dgZruxfnJngbb7pjKKW21SlUmZGgUXxGS+z+iRF fyQbHZQTpSG0lGTDTD6uY4uKISXxgprtoiaoFKR1stNVfpFaE1cmlkWtWFEg5yZ30TEG uFlX+o5oc23dHFHbhtXiYG18gpQzY6b9UwKoGTYDKsztbVlZVTf6n81yxjshIJJGPyHS Tf/TJrACsojNaXxC53Bw4fRLTUC3W1l9lnETSzthl9z/GWmTGlEx0cueG9BzDDBzienH Y1Vw==; dara=google.com ARC-Authentication-Results: i=1; gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Received: from shymkent.ilbers.de (shymkent.ilbers.de. [85.214.156.166]) by gmr-mx.google.com with ESMTPS id 4fb4d7f45d1cf-66315322b64si45746a12.7.2026.03.11.04.15.37 for (version=TLS1_2 cipher=ECDHE-ECDSA-CHACHA20-POLY1305 bits=256/256); Wed, 11 Mar 2026 04:15:37 -0700 (PDT) Received-SPF: pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) client-ip=85.214.156.166; Received: from user-B660.promwad.corp ([159.148.83.123]) (authenticated bits=0) by shymkent.ilbers.de (8.15.2/8.15.2/Debian-8+deb9u1) with ESMTPSA id 62BBFXNi022597 (version=TLSv1.2 cipher=ECDHE-RSA-AES256-GCM-SHA384 bits=256 verify=NOT); Wed, 11 Mar 2026 12:15:35 +0100 From: Anton Mikanovich To: isar-users@googlegroups.com Cc: Joshua Watt , Richard Purdie , Felix Moessbauer Subject: [PATCH v3 2/2] bitbake: Remove custom exception backtrace formatting Date: Wed, 11 Mar 2026 13:15:26 +0200 Message-Id: <20260311111526.665094-3-amikan@ilbers.de> X-Mailer: git-send-email 2.34.1 In-Reply-To: <20260311111526.665094-1-amikan@ilbers.de> References: <20260311111526.665094-1-amikan@ilbers.de> MIME-Version: 1.0 X-Spam-Status: No, score=-4.6 required=5.0 tests=DKIMWL_WL_MED,DKIM_SIGNED, DKIM_VALID,DKIM_VALID_EF,HEADER_FROM_DIFFERENT_DOMAINS, MAILING_LIST_MULTI,RCVD_IN_DNSWL_BLOCKED,RCVD_IN_MSPIKE_H3, RCVD_IN_MSPIKE_WL,RCVD_IN_RP_CERTIFIED,RCVD_IN_RP_RNBL,RCVD_IN_RP_SAFE, SPF_PASS autolearn=unavailable autolearn_force=no version=3.4.2 X-Spam-Checker-Version: SpamAssassin 3.4.2 (2018-09-13) on shymkent.ilbers.de X-Original-Sender: amikan@ilbers.de X-Original-Authentication-Results: gmr-mx.google.com; spf=pass (google.com: domain of amikan@ilbers.de designates 85.214.156.166 as permitted sender) smtp.mailfrom=amikan@ilbers.de Content-Type: text/plain; charset="UTF-8" Precedence: list Mailing-list: list isar-users@googlegroups.com; contact isar-users+owners@googlegroups.com List-ID: X-Spam-Checked-In-Group: isar-users@googlegroups.com X-Google-Group-Id: 914930254986 List-Post: , List-Help: , List-Archive: , List-Unsubscribe: , X-TUID: tLFehwnZauzT From: Joshua Watt Backport of upstream bitbake patch c25e7ed128b9fd5b53d28d678238e2f3af52ef8b. Removes the code in bitbake to show custom backtrace formatting for exceptions. In particular, the bitbake exception code prints function arguments, which while helpful is a security problem when passwords and other secrets can be passed as function arguments. As it turns out, the handling of the custom serialized exception stack frames was pretty much made obsolete by d7db75020ed ("event/msg: Pass formatted exceptions"), which changed the events to pass a preformatted stacktrack list of strings, but the passing of the serialized data was never removed. Change all the code to use the python traceback API to format exceptions instead of the custom code; conveniently traceback.format_exception() also returns a list of stack trace strings, so it can be used as a drop in replacement for bb.exception.format_exception() [Felix: adjusted to bitbake path in isar repo] Signed-off-by: Joshua Watt Signed-off-by: Richard Purdie Signed-off-by: Felix Moessbauer --- bitbake/lib/bb/cooker.py | 32 +++++++++--- bitbake/lib/bb/event.py | 9 +--- bitbake/lib/bb/exceptions.py | 96 ----------------------------------- bitbake/lib/bb/msg.py | 4 -- bitbake/lib/bb/ui/teamcity.py | 5 -- 5 files changed, 25 insertions(+), 121 deletions(-) delete mode 100644 bitbake/lib/bb/exceptions.py diff --git a/bitbake/lib/bb/cooker.py b/bitbake/lib/bb/cooker.py index c5bfef55..701cf51b 100644 --- a/bitbake/lib/bb/cooker.py +++ b/bitbake/lib/bb/cooker.py @@ -17,7 +17,7 @@ import threading from io import StringIO, UnsupportedOperation from contextlib import closing from collections import defaultdict, namedtuple -import bb, bb.exceptions, bb.command +import bb, bb.command from bb import utils, data, parse, event, cache, providers, taskdata, runqueue, build import queue import signal @@ -2098,7 +2098,6 @@ class Parser(multiprocessing.Process): except Exception as exc: tb = sys.exc_info()[2] exc.recipe = filename - exc.traceback = list(bb.exceptions.extract_traceback(tb, context=3)) return True, None, exc # Need to turn BaseExceptions into Exceptions here so we gracefully shutdown # and for example a worker thread doesn't just exit on its own in response to @@ -2299,8 +2298,12 @@ class CookerParser(object): return False except ParsingFailure as exc: self.error += 1 - logger.error('Unable to parse %s: %s' % - (exc.recipe, bb.exceptions.to_string(exc.realexception))) + + exc_desc = str(exc) + if isinstance(exc, SystemExit) and not isinstance(exc.code, str): + exc_desc = 'Exited with "%d"' % exc.code + + logger.error('Unable to parse %s: %s' % (exc.recipe, exc_desc)) self.shutdown(clean=False) return False except bb.parse.ParseError as exc: @@ -2309,20 +2312,33 @@ class CookerParser(object): self.shutdown(clean=False, eventmsg=str(exc)) return False except bb.data_smart.ExpansionError as exc: + def skip_frames(f, fn_prefix): + while f and f.tb_frame.f_code.co_filename.startswith(fn_prefix): + f = f.tb_next + return f + self.error += 1 bbdir = os.path.dirname(__file__) + os.sep - etype, value, _ = sys.exc_info() - tb = list(itertools.dropwhile(lambda e: e.filename.startswith(bbdir), exc.traceback)) + etype, value, tb = sys.exc_info() + + # Remove any frames where the code comes from bitbake. This + # prevents deep (and pretty useless) backtraces for expansion error + tb = skip_frames(tb, bbdir) + cur = tb + while cur: + cur.tb_next = skip_frames(cur.tb_next, bbdir) + cur = cur.tb_next + logger.error('ExpansionError during parsing %s', value.recipe, exc_info=(etype, value, tb)) self.shutdown(clean=False) return False except Exception as exc: self.error += 1 - etype, value, tb = sys.exc_info() + _, value, _ = sys.exc_info() if hasattr(value, "recipe"): logger.error('Unable to parse %s' % value.recipe, - exc_info=(etype, value, exc.traceback)) + exc_info=sys.exc_info()) else: # Most likely, an exception occurred during raising an exception import traceback diff --git a/bitbake/lib/bb/event.py b/bitbake/lib/bb/event.py index 4761c868..952c85c0 100644 --- a/bitbake/lib/bb/event.py +++ b/bitbake/lib/bb/event.py @@ -19,7 +19,6 @@ import sys import threading import traceback -import bb.exceptions import bb.utils # This is the pid for which we should generate the event. This is set when @@ -759,13 +758,7 @@ class LogHandler(logging.Handler): def emit(self, record): if record.exc_info: - etype, value, tb = record.exc_info - if hasattr(tb, 'tb_next'): - tb = list(bb.exceptions.extract_traceback(tb, context=3)) - # Need to turn the value into something the logging system can pickle - record.bb_exc_info = (etype, value, tb) - record.bb_exc_formatted = bb.exceptions.format_exception(etype, value, tb, limit=5) - value = str(value) + record.bb_exc_formatted = traceback.format_exception(*record.exc_info) record.exc_info = None fire(record, None) diff --git a/bitbake/lib/bb/exceptions.py b/bitbake/lib/bb/exceptions.py deleted file mode 100644 index 801db9c8..00000000 --- a/bitbake/lib/bb/exceptions.py +++ /dev/null @@ -1,96 +0,0 @@ -# -# Copyright BitBake Contributors -# -# SPDX-License-Identifier: GPL-2.0-only -# - -import inspect -import traceback -import bb.namedtuple_with_abc -from collections import namedtuple - - -class TracebackEntry(namedtuple.abc): - """Pickleable representation of a traceback entry""" - _fields = 'filename lineno function args code_context index' - _header = ' File "{0.filename}", line {0.lineno}, in {0.function}{0.args}' - - def format(self, formatter=None): - if not self.code_context: - return self._header.format(self) + '\n' - - formatted = [self._header.format(self) + ':\n'] - - for lineindex, line in enumerate(self.code_context): - if formatter: - line = formatter(line) - - if lineindex == self.index: - formatted.append(' >%s' % line) - else: - formatted.append(' %s' % line) - return formatted - - def __str__(self): - return ''.join(self.format()) - -def _get_frame_args(frame): - """Get the formatted arguments and class (if available) for a frame""" - arginfo = inspect.getargvalues(frame) - - try: - if not arginfo.args: - return '', None - # There have been reports from the field of python 2.6 which doesn't - # return a namedtuple here but simply a tuple so fallback gracefully if - # args isn't present. - except AttributeError: - return '', None - - firstarg = arginfo.args[0] - if firstarg == 'self': - self = arginfo.locals['self'] - cls = self.__class__.__name__ - - arginfo.args.pop(0) - del arginfo.locals['self'] - else: - cls = None - - formatted = inspect.formatargvalues(*arginfo) - return formatted, cls - -def extract_traceback(tb, context=1): - frames = inspect.getinnerframes(tb, context) - for frame, filename, lineno, function, code_context, index in frames: - formatted_args, cls = _get_frame_args(frame) - if cls: - function = '%s.%s' % (cls, function) - yield TracebackEntry(filename, lineno, function, formatted_args, - code_context, index) - -def format_extracted(extracted, formatter=None, limit=None): - if limit: - extracted = extracted[-limit:] - - formatted = [] - for tracebackinfo in extracted: - formatted.extend(tracebackinfo.format(formatter)) - return formatted - - -def format_exception(etype, value, tb, context=1, limit=None, formatter=None): - formatted = ['Traceback (most recent call last):\n'] - - if hasattr(tb, 'tb_next'): - tb = extract_traceback(tb, context) - - formatted.extend(format_extracted(tb, formatter, limit)) - formatted.extend(traceback.format_exception_only(etype, value)) - return formatted - -def to_string(exc): - if isinstance(exc, SystemExit): - if not isinstance(exc.code, str): - return 'Exited with "%d"' % exc.code - return str(exc) diff --git a/bitbake/lib/bb/msg.py b/bitbake/lib/bb/msg.py index 3e18596f..4f616ff4 100644 --- a/bitbake/lib/bb/msg.py +++ b/bitbake/lib/bb/msg.py @@ -89,10 +89,6 @@ class BBLogFormatter(logging.Formatter): msg = logging.Formatter.format(self, record) if hasattr(record, 'bb_exc_formatted'): msg += '\n' + ''.join(record.bb_exc_formatted) - elif hasattr(record, 'bb_exc_info'): - etype, value, tb = record.bb_exc_info - formatted = bb.exceptions.format_exception(etype, value, tb, limit=5) - msg += '\n' + ''.join(formatted) return msg def colorize(self, record): diff --git a/bitbake/lib/bb/ui/teamcity.py b/bitbake/lib/bb/ui/teamcity.py index fca46c28..7eeaab8d 100644 --- a/bitbake/lib/bb/ui/teamcity.py +++ b/bitbake/lib/bb/ui/teamcity.py @@ -30,7 +30,6 @@ import bb.build import bb.command import bb.cooker import bb.event -import bb.exceptions import bb.runqueue from bb.ui import uihelper @@ -102,10 +101,6 @@ class TeamcityLogFormatter(logging.Formatter): details = "" if hasattr(record, 'bb_exc_formatted'): details = ''.join(record.bb_exc_formatted) - elif hasattr(record, 'bb_exc_info'): - etype, value, tb = record.bb_exc_info - formatted = bb.exceptions.format_exception(etype, value, tb, limit=5) - details = ''.join(formatted) if record.levelno in [bb.msg.BBLogFormatter.ERROR, bb.msg.BBLogFormatter.CRITICAL]: # ERROR gets a separate errorDetails field -- 2.34.1 -- You received this message because you are subscribed to the Google Groups "isar-users" group. To unsubscribe from this group and stop receiving emails from it, send an email to isar-users+unsubscribe@googlegroups.com. To view this discussion visit https://groups.google.com/d/msgid/isar-users/20260311111526.665094-3-amikan%40ilbers.de.