庆祝我们的2021年开源贡献 - The Trail of Bits博客

Sam Moelius
2021年12月31日
年度回顾

在Trail of Bits，我们以将最佳工具开源而自豪，例如algo、manticore和graphtage。但虽然这篇文章关于开源，它并不关于我们的工具……

2021年，Trail of Bits员工提交了超过190个拉取请求（PRs），这些请求被合并到非Trail of Bits仓库中。这展示了我们保护整个软件生态系统的承诺，以及为每个人提高软件质量的努力。贡献的代表性列表出现在文章末尾，但以下是一些亮点：

• LLVM是一组编译器和工具链技术。LLVM作为许多流行编译器的后端，如clang、rustc和swiftc。我们为LLVM中的错误实现了多项修复，包括纠正文档错误、确保在clang的AST转储模式下生成有效的JSON，并确保LLVM仅接受格式良好的位码。
• Nixpkgs是一个包含超过80,000个软件包的集合，可以通过Nix包管理器安装。我们对许多广泛使用的Nix包进行了改进和错误修复，包括Go、Hevm、libff、Protobuf和SBV。
• Osquery是一个基于SQL的操作系统检测、监控和分析框架。我们对osquery做出了众多贡献，最显著的是基于新的Endpoint Security API为macOS添加进程事件监控；彻底 overhaul 项目的代码签名、打包和CI；最后但同样重要的是，添加对Apple Silicon的原生支持，这是Apple今年开始过渡的基于ARM的架构。
• Python是一种解释型、高级、通用编程语言。我们为Python打包/分发生态系统中的关键包贡献了一系列修复和新功能，包括mypy、pip-api和Warehouse。我们还在pyelftools（主导的Python ELF解析器）中添加了DWARFv5支持。
• Pwndbg是一个GDB插件，使使用GDB调试“不那么糟糕”。我们在从命令解析到匿名页面映射方式的各个方面对pwndbg进行了改进和错误修复。

我们想承认，提交PR只是开源体验的一小部分。有人必须审查PR。有人在PR合并后必须维护代码。早期PR的提交者必须编写测试以确保其代码功能的保留。

我们贡献这些项目部分是因为我们热爱这门手艺，但也因为我们发现这些项目有用。为此，我们向开源社区致以最诚挚的感谢，并祝愿每个人2022年快乐、安全和富有成效！

Osquery描述于2022年1月3日更新。

Trail of Bits的2021年开源贡献部分列表

• assert-rs/assert_cmd
  Add try_ variants of Assert methods #128
  feat: Refine append_context bounds #130

• aws/amazon-ecs-agent
  prevents a goroutine from being leaked if a timeout occurs when calling forceCloseConnection #2854

• Azure/azure-container-networking
  fix: prevents a goroutine from being leaked in internalapi.go #850

• cdisselkoen/llvm-ir
  Add llvm_version function (resolves #13) #14
  Add llvm-13 case to llvm_version #17

• CycloneDX/cyclonedx-python-lib
  model/vulnerability: fix optional type #61

• dapphub/dapptools
  Update hevm deps and nixpkgs to 21.05 #655

• di/pip-api

  Use pip list’s JSON output for installed_distributions #93
  pip_api: type hints #97
  Allow requirement markers to be parsed #99
  Allow installed_distributions to be filtered for global distributions #103
  Add support for parsing URL requirements #109
  Support the –path parameter when calling pip list #112
  pip_api/_call: pass PIP_DISABLE_PIP_VERSION_CHECK to all invocations #114

• eliben/pyelftools
  dwarf: initial DWARFv5 support #363

• ESultanik/visie
  Code cleanup #1

• firemark/pixelopolis
  Support bsd/posix invocation #1

• Gallopsled/pwntools
  Fix #1966: Add arch alias: x86-64 -> amd64 #1967

• GaloisInc/FAW
  Adds a Polytracker File Detail View Plugin #35

• haampie/libtree
  Fix integration test when using CMake Ninja generator #33

• icedland/iced
  Indicate whether an instruction is a “string” instruction #186

• iovxw/gleipnir
  Fix rpc server permissions #238

• kgabis/parson
  Fix memleak when parsing keys with embedded null bytes #157

• kubernetes/minikube
  Goroutine leak fix #11247

• LLVM
  [BitcodeAnalyzer] allow a motivated user to dump BLOCKINFO (D107536)
  [clang] Fix JSON AST output when a filter is used (D108441)
  [docs] [NFC] Clarify the datalayout documentation (D108962)
  [BitcodeReader] fix a logic error in vector type element validation (D109655)

• microsoft/hcsshim
  prevents a goroutine from being leaked if binary cmd fails to finish #993

• microsoft/vcpkg-tool
  binarycaching: Add NuGet timeout configuration entry #95

• microsoft/vcpkg
  [vcpkg_configure_make] MacOS assume target arch is host arch #18632
  [docs] Describe nugettimeout option in binarycaching #19084

• NixOS/nixpkgs
  echidna: init at 1.7.2 #106919
  pe-parse: init at 1.2.0 #107506
  liquidctl: init at 1.4.2 #108258
  python3Packages.slither-analyzer: 0.6.14 -> 0.7.0 #108610
  uthenticode: init at 1.0.4 #109378
  pythonPackages.manticore: fix tests on darwin #112069
  nxpmicro-mfgtools: 1.4.43 -> 1.4.72 #113516
  sgx-sdk: init at 2.14 #126990
  python3Packages.crytic-compile: 0.1.13 -> 0.2.0 #130241
  haskellPackages.hevm: unbreak #131059
  solc-select: init at 0.2.1 #131943
  protobuf: 3.18.0 -> 3.19.0 #142096
  go: use tzdata from Nix on Darwin #142494
  slither-analyzer: 0.8.1 -> 0.8.2 #150058
  libff: fix build on aarch64 #150850
  haskellPackages.sbv: fix build on aarch64 #150855

• nodejs/node
  http2: fix double free due to handling of rst_stream with cancel code #39423
  http2: update handling of streams on rst_stream frames #39622

• osquery/osquery
  Remove unused ev2 code #6878
  Remove unused/experimental ebpf code #6879
  Fix heap-use-after-free in deregisterEventSubscriber #6880
  Fix UB and dangerous casting in the pubsub framework #6881
  CI: Add support for GitHub Actions #6885
  Reduce the compilation units from libarchive #6886
  Fix a leak in libdpkg when querying the deb_packages table #6892
  [macOS][CI] Update XCode to 12.3 and Update min macOS version to 10.12 #6896
  Fix data type macro used for 64-bit timestamp variables #6897
  Disable incremental linking to reduce build size on Windows #6898
  Spellcheck and Markdown nits #6899
  Remove unused tests for Rocksdb and Inmemory db plugins #6900
  Fix typos across source code #6901
  Change libdpkg submodule url to our own github mirror #6903

  CMake: Add -pthread compile option on posix platforms #6909
  Disable deprecated TLS versions 1.0, 1.1 #6910
  GitHub Actions: Use Xcode 12.3, SDK 10.12 #6913
  Significantly speed up CMake configuration phase #6914
  Add column for system extensions managed by configuration policy (system_extensions table) #6915
  Rename yara str functions to avoid symbol collisions #6917
  Remove unused empty test file #6918
  GitHub Actions: Fix .deb artifacts, add scheduled builds #6920
  Move packaging logic to osquery-packaging #6921
  Fix SystemControlsTest adding sunrpc as an expected subsystem #6932
  Docs: fix reference to a Powershell script on Windows #6936
  Fix StartupItemTest failing due to unexpected values #6940
  Fix XattrTests failing due to unexpected attribute name #6941
  Fix ExtendedAttributesTableTests failing due to an unexpected attribute #6942
  Fix an incorrect check in StartupItems test #6950
  Improve explanations of event control flags #6954
  Update the Linux install steps and package listing #6956
  Update the info about osquery’s TLS version support #6963
  Fix mem leak regression with Windows’ sids API #6984
  Always use BIGINT macro for ‘long long’ data #6986
  Make Group ID columns consistent across Windows tables #6987
  Docs: change reference about Azure Pipelines to GitHub Actions #6988
  [packaging] Remove extraneous lenses directory for augues on macOS #6998
  Docs: add a note on enabling Windows to build with CMake’s long paths #7010
  libs: Update OpenSSL to version 1.1.1k #7026
  Correct docs about OpenSSL and TLS behavior #7033
  Remove Buck leftovers that supported building with old versions of OpenSSL #7034
  Correct the example in the windows_events table spec #7035
  Improve docs on FIM, mention NTFS and Audit, etc. #7036
  Add an option to enable incremental linking on Windows #7044
  [macOS] EndpointSecurity based process events #7046
  Docs: add a security assurance case #7048
  Fix tls_enroll_max_attempts flag name in the documentation #7049
  Use standalone CPack packaging #7059
  Correct RocksDB error code and subcode printing on open failure #7069
  Print extension sdk minimum version required when failing to load #7074
  Fix extensions crash on shutdown #7075
  Improve speed of osquery shutdown procedure #7077
  Remove duplicated osquery_utils_aws_tests-test #7078
  CI: Regenerate sccache cache when compiler version changes #7081
  [AWS] Add support for IMDSv2 (Instance Metadata service) #7084
  docs: Update process auditing requirements #7102
  Improve shutdown speed during initialization #7106
  Watchdog should wait for the worker to shutdown #7116
  chrome_extensions: Compute the identifier from the ‘key’ property #7124
  Implement infinite enrollment retries #7125
  Remove POSIX-only -fexceptions flag on Windows #7126
  Fix crash and deadlocks in the support for recursive logging #7127
  Minor cleanup of unused variables #7128

  Docs: bring the YARA wiki page up to date #7172
  libs: Update the ebpfpub library #7173
  [libs][yara] enable and compile the macho module on macOS #7174
  Fix choco not failing when an error occurs during install or upgrade #7182
  Fix broadcasting empty logs to logger plugins #7183
  Update macOS build to include app bundle related files #7184
  libs: Update Strawberry Perl to 5.32.1.1, use HTTPS downloads #7199
  Prevent race condition between shutdown and worker or extension launch #7204
  [AWS] Optionally enable debug option and restrict content-type header size for PUT req #7216
  libs: Update ebpfpub #7219
  Fix osquery_info build_platform column value on Linux #7254
  [macOS][packaging] Update the packaging repo commit for #7236 related fixes #7255
  [macOS][packaging] Create an app bundle along with other package_data #7263
  audit: socket_events improvements #7269
  [linux][packaging] Update packaging paths #7271
  Change logger_mode flag to be actually interpreted as an octal #7273
  Update packaging SHA #7279
  Update osquery installed artifacts default paths in code #7285
  Update osquery installed artifacts paths in the documentation #7286
  macos path fix in launchd plist #7288
  Correct macOS installed app bundle path in osqueryctl and doc #7289
  libs: Update OpenSSL to version 1.1.1l #7293
  Prevent osquery from killing itself when the –force flag is used #7295
  bpf: Improve publisher reliability #7302
  docs: update macOS ESF documentation #7303
  Update installation guide to use newer macOS paths #7311
  Fix ASL test on macOS 11 and later #7320
  Apple Silicon support #7330
  Avoid string copies when looping through cron search dirs #7331
  Update the CI Linux Docker image #7332
  Windows: Detect when an extension has not started #7355
  Skip deprecated ASL test when targeting 10.13+ SDK #7358

  Respect read_max flag when hashing using ssdeep #7367
  Restore query packs in Windows packaging #7388
  Fix crash when windows_security_products errors out #7401
  CI: Update packaging commit to fix Linux symlinks #7404
  Prevent running discovery queries when fuzzing #7418
  Fix how we disable tables in the fuzzer init method #7419
  Fix linking of thirdparty_sleuthkit #7425
  Update sqlite to version 3.37.0 #7426

• paritytech/substrate
  node-template: remove redundant types from runtime #9161

• pwndbg/pwndbg
  format_args: display fd path #825
  Fix #858 #877
  Fix #881 #883
  vmmap: name anonymous pages #933
  Fix #946 context when reg value deref fails #948
  Add memoize command for toggling caching, useful for debugging pwndbg #951
  Add attachp command #965
  Remove shebang and coding lines #972
  Remove Py2 class object inheritance #973
  Fix #932,#788: fix command parsing #974
  Skip attachp tests when cant attach #975
  Fix #932,#788: fix command parsing #976

• pypa/warehouse
  api-reference/json: document vulnerabilities in responses #10431

• pysmt/pysmt
  Fix to correctly pass logic to solvers started by Portfolio #683

• python/mypy
  mypy/build: Use _load_json_file in load_tree #11575

• rust-fuzz/afl.rs
  Expand CARGO environment variable at runtime #184
  Test with both stable and nightly in CI #194
  Handle old LLVM pass manager on rustc 1.57 #197

• rust-lang/rust-clippy
  Add format_in_format_args and to_string_in_format_args lints #7743
  Fix #7903 #7906
  Add unnecessary_to_owned lint #7978

• rust-lang/rust
  Update Clippy dependencies without patch versions #88517
  Implement #85440 (Random test ordering) #89082
  Pass real crate-level attributes to pre_expansion_lint #89214

• rustsec/advisory-db
  parse_duration: parse DoS through payloads with big exponent #827

• samuelcolvin/pydantic
  doc(schema): fix a callout #2620

• Smithay/udev-rs
  lib, device: begin using list::List instead of custom structs #22

• solana-labs/rbpf
  Fix verifier shift instruction overflows imm value #212

• SRI-CSL/gllvm
  extractor: Make extraction errors fatal #37
  Don’t treat -w/-W as compile-only indicators #43
  Support LLVM_LINK_FLAGS #51
  extractor, utils: dedupe bitcode paths before linking #54
  get-bc: tweak LogInfo message #55

• taiki-e/cargo-llvm-cov
  Implement –failure-mode option #91
  Use –target-dir in favor of CARGO_TARGET_DIR #112

• WLBF/single-instance
  Use an abstract namespace UDS on Linux #7

• **