Skip to content

Binary data, compression and identifiers#

This page is about bytes: holding them, encoding them as text, compressing, archiving, and generating identifiers. Reading and writing files is covered in Files, paths and streams, text in Working with text.

Holding bytes#

AByteBuffer is a growing array of bytes. Fill it with operator<< (it is an IOutputStream) or by constructing it from a pointer and size, and get a non-owning window into it with slice().

AByteBuffer buffer;
buffer << std::uint32_t(0xcafebabe) << "tail";       // raw bytes of the number, then the string

AByteBufferView head = buffer.slice(0, 4);           // first 4 bytes, no copy
AByteBuffer copy = AByteBuffer(head);                // own copy

Warning

Numbers are written as is, in the native byte order. If you exchange data between machines, agree on an encoding.

A buffer can be constructed from a string with AByteBuffer::fromString(), and read back as text with AString::fromUtf8(buffer). For a different text encoding, use the AString(AByteBufferView, AStringEncoding) constructor and AString::encode().

Hex and Base64#

1
2
3
4
5
6
std::string_view text = "Building beautiful programs in pure C++ without chromium embedded framework";
AByteBuffer buffer(text.data(), text.size());
auto base64 = buffer.toBase64String();
EXPECT_EQ(base64, "QnVpbGRpbmcgYmVhdXRpZnVsIHByb2dyYW1zIGluIHB1cmUgQysrIHdpdGhvdXQgY2hyb21pdW0gZW1iZWRkZWQgZnJhbWV3b3Jr");
auto decoded = AString::fromUtf8(AByteBuffer::fromBase64String(base64));
EXPECT_EQ(decoded, text);

toHexString() and AByteBuffer::fromHexString() work the same way for hexadecimal.

Compression#

aui::zlib::compress and aui::zlib::decompress from AUI/Util/LZ.h convert between AByteBuffers (both append to the destination buffer).

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
auto source = makeSource();

AByteBuffer compressed;
aui::zlib::compress(source, compressed);

EXPECT_LE(compressed.size(), source.size());

AByteBuffer decompressed;
aui::zlib::decompress(compressed, decompressed);

EXPECT_EQ(source, decompressed);

If the unpacked data is large, aui::zlib::decompressToStream returns an IInputStream which decompresses on the fly, so you can process it chunk by chunk.

ZIP archives#

AUI/Util/Archive.h reads and writes ZIP files.

To create an archive, wrap an AFileOutputStream (any seekable output stream works) in a aui::archive::zip::Writer and add files with a lambda that writes their contents:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
12
13
14
15
{
    aui::archive::zip::Writer writer(std::make_unique<AFileOutputStream>("test.zip"));
    writer.openFileInZip(APath("test") / "test.txt", [](IOutputStream& os) {
        os << std::string_view("1234\n");
    });
}

bool called = false;
aui::archive::zip::read(AFileInputStream("test.zip"), [&](const aui::archive::FileEntry& e) {
    EXPECT_FALSE(called);
    called = true;
    EXPECT_EQ(e.name, "test/test.txt");
    EXPECT_EQ(AString(AByteBuffer::fromStream(e.open()), AStringEncoding::LATIN1), "1234\n");
});
EXPECT_TRUE(called);

To read, call aui::archive::zip::read with a seekable input stream and a visitor. It is called for each entry, and the entry's data is available through open(). Don't use the stream after the visitor returns.

aui::archive::zip::read(AFileInputStream("archive.zip"), [](const aui::archive::FileEntry& entry) {
    ALogger::info("Zip") << entry.name;
    auto data = AByteBuffer::fromStream(entry.open());
});

To simply unpack everything to a folder, pass the aui::archive::ExtractTo visitor:

aui::archive::zip::read(AFileInputStream("archive.zip"), aui::archive::ExtractTo { .prefix = "unpacked" });

UUIDs and random data#

AUuid parses and prints UUIDs in canonical and raw formats and throws AUuidException on malformed input.

1
2
3
AString s = "123e4567-e89b-12d3-a456-426655440000";
AUuid uuid = s;
ASSERT_EQ(uuid.toString(), s);

Use ARandom to generate new identifiers or random bytes:

ARandom random;
AUuid id = random.nextUuid();
AByteBuffer nonce = random.nextBytes(16);

Note

ARandom is not a cryptographically secure generator. For secrets, hashing and encryption use aui::crypt.