JOSTALY TECHNOLOGIES
12/09/2024

142f308c2c67b97f47a397a3f08bd652ca94a73b5bfaef4d98dc80aba41f6553  ./xz.article.git.tgz
d91c0e8d6acca91a8fef877666f4985f07427d1f5899502bd3115a42c2501994  ./xz.article.git.tgz.asc

"Our lives begin to end the day we become silent about things that matter."
	- Martin Luther King

0) NEWS OF INTEREST
30 Aug : CVE-2023-22527 - Atlassian Confluence
04 Sep : Haliburton data breach

(CVE-2024-3094): Backdoor in XZ utils configure

1) ARTICLE

	This article digs into augmenting xz and then replacement of standard libc.
Xz was a slightly larger endeavor as it's split into liblzma, lzmainfo, xz,
xzdec, and lzmadec. The article was sort of a race against time to generate
content for the company website, double as a development task/testing, triple
for testing/bug-hunting, but there's only so much time.

	The standard configuration for most software packages is complex to say the
least, and gets even more complicated from nefarious actors adding backdoors in
configuration scripts, the build system, and (in)advertent security issues baked
into the software itself. The complexity of these systems makes finding the
issues akin to the proverbial needle in the haystack; sometimes even bordering
on the line of obfuscation if not crossing it (under the hood at least).

	After running a configuration, build, and standard make install to a
modified --prefix, the entire build system was ripped out and replaced with
shell scripts of what is actually getting built and modifying some compiler
flags. The resulting install files then linked back into the source tree, the
default system versions uninstalled / removed. See the makefiles in each
directory. They run a build.sh shell script.  XZ has testing, next was to
ensure that all tests work with the simplified build. A lot of work goes into
open source software, most times it's used without an appreciation or
understanding the mechanics.

	Obsessive on code is something JOSTALY TECHNOLOGIES is good at. Our libc
variant hooks into our patent pending design and while the prototypes remain the
same to be compliant with the standard, there's a number of differences that
narrow in on code sections that are interesting. Complexity is generally the
enemy of something being straight-forward and easy to understand, simple sticks.

	A top down approach to start the augmentation/replacement was used
initially due to the smaller number of source files. Almost immediately however,
a segfault was encountered with the application layer releasing memory allocated
by the liblzma library, causing a conflict between the two libc memory managers,
introducing a chicken and the egg problem. A temporary disable of linking with
the library and direct library source file compilation was required to bypass
the problem until all components had been augmented. The lzmainfo application
was first, xzdec/lzmadec second, and finally xz itself. It's also worthwhile to
point out that lzmadec is really just xzdec.

	Immediately with augmentation/replacement a missing null check was found in
lzma_free:
<code>
    extern void
    lzma_free(void *ptr, const lzma_allocator *allocator)
    {
        if (allocator != NULL && allocator->free != NULL)
            allocator->free(allocator->opaque, ptr);
    ->  else if (ptr != NULL)
            free(ptr);
   
        return;
    }
</code>

	Not very interesting. JOSTALY TECHNOLOGIES believes in pedantic error
checking though, the NULL page is a popular exploitation, checking pointers
everywhere can uncover interesting behavior.

	The stdio.h header is questionable, it provides essentially a middle man
interface around the unistd.h raw syscalls. It's very common to see the standard
f(write|read|close|error|open) calls, etc. so this interface is useful to
implement avoiding translation to unistd.h variants. Our implementation is
essentially a thin direct pass-through to the raw syscalls to avoid the
"middle-man" as much as possible. Unless required by the standard, the use of
global variables are not used, and modifications made to accommodate this.

	String manipulation is also a popular exploitation known for causing issues.
Most programmers or anyone familiar with the standard C formatting will notice
the output has raw % formatting and the arguments printed after the format
string. This avoids string manipulation code paths at the expense of aesthetics
and possible detection of manipulation failures within the formatting.

	It's always interesting to dig into an unfamiliar code base, and the
disorientation that comes along with it. This assembler file caught my eye:

<code>
    LZMA_CRC64:
        _CET_ENDBR
...
        pushl   %ebx
        pushl   %esi
        pushl   %edi
        pushl   %ebp
        movl    0x14(%esp), %esi /* buf */
        movl    0x18(%esp), %edi /* size */
        movl    0x1C(%esp), %eax /* crc LSB */
</code>

	While you would have to configure xz with a 32-bit environment (host_cpu=i?86)
to turn on getting this built in, and then use a 64-bit application to call that
function, the address would be cut in half. It was questionable, took a little bit
of time to track down, so it was just removed along with other sections.
Particularly the BCJ (Branch/Conditional/Jump) coders/decoders, these are sort of
interesting as they're modifying instructions for particular architectures for
better compression results. Tests were modified to accommodate these changes. Raw
assembly files were ripped out, arguments ripped out that are more development
type of tools (--list, --block-size, --block-list) that are probably better
off as an independent tool. A number of OS specific macros were additionally
removed to simplify down to generic (ENABLE_SANDBOX, etc.). Multi-threading
encoders/decoders were also removed. Given the speed of current processors, the
assembler and threading have a feel of premature optimization and extra
complexity.

	All in all, this xz variant is a bit less complex, doesn't require a build
system, and is augmented with our operating system / library routines. Each of
the build scripts have two important toggles:

		* CFLAGS	: Toggle between our backend implementation of libc, or use
	the development system libc.
		* LZMA_OBJS	: Toggle between direct liblzma sources, or use static
	linking of liblzma.

	Interesting limitations on our end that were found in addition to above:

	-fstack-protector strong:
		A simple implementation of the hidden stack_chk_fail emission call is
	not sufficient. Internally Thread Local Storage (TLS) is used with segment
	registers to achieve this (on the x86). This is essentially hooking into a
	backend compiler feature. This may differ between compilers, and results
	may vary. Local variables of relevant size are generally avoided, a more
	granular approach would be nice in addition to our other security features.