Hypercore

Hypercore is a secure, distributed append-only log for sharing large datasets and realtime streams. It supports sparse replication, verified reads, optional block encryption, and session-based workflows for building higher-level local-first and peer-to-peer data structures.

Install

npm i hypercore

Quickstart

import Hypercore from 'hypercore'

const core = new Hypercore('./my-first-core', { valueEncoding: 'utf-8' })
await core.ready()

await core.append('hello')
await core.append('from hypercore')

console.log('length:', core.length)
console.log('first block:', await core.get(0))
console.log('second block:', await core.get(1))

for await (const block of core.createReadStream()) {
  console.log('streamed:', block)
}

await core.close()

API Reference

Constructors and Sessions

new Hypercore(storage, [key], [options])

• Signature: new Hypercore(storage, [key], [options])
• Parameters:
  • storage: a directory path, a hypercore-storage instance, or a storage value supplied through options.storage.
  • key: optional public key used to reopen an existing core. If omitted, Hypercore loads the key from storage or creates a new writable core.
  • options: optional settings for creation and runtime behavior, including:
    • createIfMissing, overwrite, force
    • valueEncoding, encodeBatch
    • keyPair, writable
    • encryption, onwait, timeout, inflightRange, notDownloadingLinger, allowFork
    • userData, manifest, preload, storage, key, ongc, onseq
• Returns: a Hypercore instance. Call await core.ready() before relying on synchronous properties such as core.key, core.discoveryKey, or core.length.
• Example: const core = new Hypercore('./data', { valueEncoding: 'json' })

User Data is a local-only key/value store with string keys and string or Buffer values. It is not replicated and is useful for peer-specific metadata such as encryption keys or app state.

The manifest controls how a core is authenticated, including version, hash, quorum, signers, prologue, linked, and manifest-level userData. Changing the manifest changes the resulting Hypercore key.

core.session([options])

• Signature: core.session([options])
• Parameters:
  • options: inherited constructor options plus session-specific fields:
    • weak: close automatically when all non-weak sessions are gone.
    • exclusive: wait for exclusive access before the session becomes ready.
    • checkout: open the core at a specific length.
    • atom: attach the session to a storage atom for atomic changes.
    • name: create a named persisted branch.
• Returns: a new Hypercore session that shares the same underlying core state.
• Example: const checkout = core.session({ checkout: 10 })

core.commit(session, opts = {})

• Signature: core.commit(session, opts = {})
• Parameters:
  • session: a session whose blocks should be committed into the default core.
  • opts: optional commit controls:
    • length: expected length after commit.
    • treeLength: expected Merkle tree length before commit.
    • keyPair: key pair used to sign the commit.
    • signature: explicit signature for the committed blocks.
• Returns: { byteLength, length } when the commit succeeds, or null if it fails.
• Example: const committed = core.commit(session)

core.snapshot([options])

• Signature: core.snapshot([options])
• Parameters:
  • options: the same options accepted by core.session([options]).
• Returns: a snapshot-backed session that does not append or truncate as the underlying core changes.
• Example: const snap = core.snapshot()

Writing and Local Mutation

await core.append(block, options = {})

• Signature: await core.append(block, options = {})
• Parameters:
  • block: a single block or an array of blocks.
  • options: append controls:
    • writable: ignore the writable guard without changing whether the core is actually writable.
    • maxLength: cap the resulting core length.
    • keyPair: key pair used to sign the appended block or batch.
    • signature: explicit signature for the append.
• Returns: { length, byteLength } for the resulting core after the append.
• Example: await core.append(['block 1', 'block 2'])

core.createWriteStream()

• Signature: core.createWriteStream()
• Parameters:
  • none.
• Returns: a writable stream that appends each written chunk as its own block.
• Example: core.createWriteStream().end('hello')

await core.clear(start, [end], [options])

• Signature: await core.clear(start, [end], [options])
• Parameters:
  • start: first block index to clear from local storage.
  • end: optional non-inclusive end index.
  • options: optional flags, currently diff to return cleared-byte information.
• Returns: metadata about cleared bytes when diff: true, otherwise null.
• Example: await core.clear(0, 10, { diff: true })

await core.truncate(newLength, [options])

• Signature: await core.truncate(newLength, [options])
• Parameters:
  • newLength: the shorter length to truncate to.
  • options: a fork number or an options object containing:
    • fork: fork id after truncation.
    • keyPair: key pair used to sign the truncation.
    • signature: explicit truncation signature.
• Returns: a promise that resolves when the truncation is persisted.
• Example: await core.truncate(4, { fork: core.fork + 1 })

Reading, Streams, and Proofs

await core.get(index, [options])

• Signature: await core.get(index, [options])
• Parameters:
  • index: block index to read.
  • options: read controls:
    • wait: wait for download if the block is missing locally.
    • onwait: callback fired when the read starts waiting.
    • timeout: maximum wait time in milliseconds.
    • activeRequests: advanced replication request handle.
    • valueEncoding: override the core's default encoding.
    • decrypt: automatically decrypt encrypted blocks.
    • raw: return undecoded data.
• Returns: the decoded block, or raw bytes when raw: true.
• Example: const first = await core.get(0, { wait: false })

await core.has(start, [end])

• Signature: await core.has(start, [end])
• Parameters:
  • start: first block index to check.
  • end: optional non-inclusive end index.
• Returns: true when all requested blocks are available locally.
• Example: const cached = await core.has(0, core.length)

await core.update([options])

• Signature: await core.update([options])
• Parameters:
  • options: optional update controls:
    • wait: wait for a merkle-tree update after peer discovery.
    • activeRequests: advanced replication request handles.
    • force: allow updates even when the core is writable.
• Returns: true if the core length changed and false otherwise.
• Example: await core.update({ wait: true })

core.update() works especially well with core.findingPeers() or a swarm flush so Hypercore knows when peer discovery is still in progress.

await core.seek(byteOffset, [options])

• Signature: await core.seek(byteOffset, [options])
• Parameters:
  • byteOffset: absolute byte offset in the logical stream.
  • options: optional seek controls:
    • wait: wait for missing data to download.
    • timeout: maximum wait time in milliseconds.
    • activeRequests: advanced replication request handles.
• Returns: [index, relativeOffset], where index is the containing block and relativeOffset is the byte offset inside that block.
• Example: const [index, offset] = await core.seek(1024)

core.createReadStream([options])

• Signature: core.createReadStream([options])
• Parameters:
  • options: stream controls:
    • start: first block index, default 0.
    • end: non-inclusive end index, default core.length.
    • wait: whether to wait for missing blocks.
    • timeout: maximum wait time.
    • live: keep the stream open for future appends.
    • snapshot: freeze end at open time when true.
• Returns: an async-iterable readable stream of decoded blocks.
• Example: for await (const block of core.createReadStream({ live: true })) {}

core.createByteStream([options])

• Signature: core.createByteStream([options])
• Parameters:
  • options: byte-range controls:
    • byteOffset: first byte to read.
    • byteLength: number of bytes to read.
    • prefetch: number of bytes to download ahead.
• Returns: an async-iterable readable stream over a byte range.
• Example: for await (const chunk of core.createByteStream({ byteOffset: 0, byteLength: 64 })) {}

await core.treeHash([length])

• Signature: await core.treeHash([length])
• Parameters:
  • length: optional logical length to hash. Defaults to the current core length.
• Returns: the Merkle tree hash for the requested length.
• Example: const hash = await core.treeHash()

await core.proof(opts)

• Signature: await core.proof(opts)
• Parameters:
  • opts: proof request options:
    • block: { index, nodes } for a block proof.
    • hash: { index, nodes } for a hash proof.
    • seek: { bytes, padding } for a seek proof.
    • upgrade: { start, length } for an upgrade proof.
• Returns: a TreeProof instance for the requested data.
• Example: const proof = await core.proof({ block: { index: 0, nodes: 0 } })

await core.verifyFullyRemote(proof)

• Signature: await core.verifyFullyRemote(proof)
• Parameters:
  • proof: a proof returned by a remote peer.
• Returns: the verified Merkle tree batch, or throws if verification fails.
• Example: const batch = await core.verifyFullyRemote(proof)

await core.signable([length], [fork])

• Signature: await core.signable([length], [fork])
• Parameters:
  • length: optional logical length to encode.
  • fork: optional fork id to encode.
• Returns: a Buffer containing the signable payload for the current key, tree hash, length, and fork.
• Example: const payload = await core.signable(core.length, core.fork)

core.download([range])

• Signature: core.download([range])
• Parameters:
  • range: optional download request:
    • start: first block index.
    • end: non-inclusive end index, or -1 for continuous downloading.
    • blocks: explicit list of block indices.
    • linear: download in order instead of randomly.
    • activeRequests: advanced replication request handles.
• Returns: a download handle with .done() and .destroy() methods.
• Example: await core.download({ start: 0, end: 10 }).done()

Extensions and Replication

core.registerExtension(name, handlers = {})

• Signature: core.registerExtension(name, handlers = {})
• Parameters:
  • name: extension name.
  • handlers: optional extension handlers:
    • encoding: compact encoding for messages.
    • onmessage: callback for messages from peers.
• Returns: an extension handle.
• Example: const ext = core.registerExtension('chat', { encoding: 'json', onmessage })

This extension API is documented as legacy. For new protocol work, the upstream README recommends creating a Protomux protocol instead.

ext.send(message, peer)

• Signature: ext.send(message, peer)
• Parameters:
  • message: encoded payload for the extension.
  • peer: a specific peer to send to.
• Returns: sends a message over the registered extension channel.
• Example: ext.send({ type: 'ping' }, peer)

ext.broadcast(message)

• Signature: ext.broadcast(message)
• Parameters:
  • message: encoded payload for all connected peers.
• Returns: broadcasts the message to every peer on the extension.
• Example: ext.broadcast({ type: 'invalidate' })

ext.destroy()

• Signature: ext.destroy()
• Parameters:
  • none.
• Returns: unregisters the extension from the core.
• Example: ext.destroy()

core.replicate(isInitiatorOrReplicationStream, opts = {})

• Signature: core.replicate(isInitiatorOrReplicationStream, opts = {})
• Parameters:
  • isInitiatorOrReplicationStream: either a boolean initiator flag or an existing Hypercore replication stream to multiplex onto.
  • opts: the same options accepted by Hypercore.createProtocolStream(), including ondiscoverykey.
• Returns: a replication stream you can pipe to a transport.
• Example: socket.pipe(core.replicate(true)).pipe(socket)

core.findingPeers()

• Signature: core.findingPeers()
• Parameters:
  • none.
• Returns: a done callback that should be called when the current peer-discovery pass finishes.
• Example: const done = core.findingPeers()

Storage Inspection and Mark-and-Sweep

await core.info([options])

• Signature: await core.info([options])
• Parameters:
  • options: optional info flags. Set storage: true to include storage byte estimates.
• Returns: an info object with fields such as key, discoveryKey, length, contiguousLength, byteLength, fork, padding, and storage.
• Example: const info = await core.info({ storage: true })

await core.startMarking()

• Signature: await core.startMarking()
• Parameters:
  • none.
• Returns: enables mark-and-sweep mode and clears any previous markings.
• Example: await core.startMarking()

await core.markBlock(start, end = start + 1)

• Signature: await core.markBlock(start, end = start + 1)
• Parameters:
  • start: first block index to mark.
  • end: optional non-inclusive end index.
• Returns: marks the requested block range so core.sweep() retains it.
• Example: await core.markBlock(10, 20)

await core.clearMarkings()

• Signature: await core.clearMarkings()
• Parameters:
  • none.
• Returns: removes all current mark-and-sweep markings.
• Example: await core.clearMarkings()

await core.sweep(opts)

• Signature: await core.sweep(opts)
• Parameters:
  • opts: optional sweep controls, currently batchSize to control how often clears flush to storage.
• Returns: clears all unmarked local blocks from storage.
• Example: await core.sweep({ batchSize: 1000 })

The README describes the full mark-and-sweep flow as startMarking(), read or manually mark the blocks you want to keep, then sweep() to clear everything else.

Lifecycle and Local Configuration

await core.close([{ error }])

• Signature: await core.close([{ error }])
• Parameters:
  • error: optional error object used to reject pending replication requests.
• Returns: fully closes the core and its active session state.
• Example: await core.close()

await core.ready()

• Signature: await core.ready()
• Parameters:
  • none.
• Returns: resolves when the core has opened and synchronous properties are populated.
• Example: await core.ready()

await core.setEncryption(encryption)

• Signature: await core.setEncryption(encryption)
• Parameters:
  • encryption: an object that satisfies the Hypercore encryption interface.
• Returns: updates the encryption implementation used by the core.
• Example: await core.setEncryption(encryption)

await core.setEncryptionKey(key, [opts])

• Signature: await core.setEncryptionKey(key, [opts])
• Parameters:
  • key: encryption key material.
  • opts: optional flags, including block to indicate a block-encryption key.
• Returns: updates the active encryption key.
• Example: await core.setEncryptionKey(key, { block: true })

core.setKeyPair(keyPair)

• Signature: core.setKeyPair(keyPair)
• Parameters:
  • keyPair: { publicKey, secretKey } buffers for the core's signer.
• Returns: updates the key pair used internally by the core.
• Example: core.setKeyPair(keyPair)

core.setActive(active)

• Signature: core.setActive(active)
• Parameters:
  • active: whether the core should stay active for download tracking and lingering replication.
• Returns: toggles activity tracking for the core.
• Example: core.setActive(false)

await core.setUserData(key, value)

• Signature: await core.setUserData(key, value)
• Parameters:
  • key: string key.
  • value: string or Buffer value.
• Returns: stores a local-only user-data entry.
• Example: await core.setUserData('encryption-key', secret)

await core.getUserData(key)

• Signature: await core.getUserData(key)
• Parameters:
  • key: string key to read.
• Returns: the stored user-data value for that key.
• Example: const secret = await core.getUserData('encryption-key')

Properties

core.writable

• Signature: core.writable
• Parameters:
  • none.
• Returns: true when the core can append or truncate. Populated after ready.
• Example: if (core.writable) await core.append('next block')

core.readable

• Signature: core.readable
• Parameters:
  • none.
• Returns: true when the core can be read from. Becomes false after close.
• Example: if (!core.readable) throw new Error('core is closed')

core.id

• Signature: core.id
• Parameters:
  • none.
• Returns: the z-base-32 string form of the public key.
• Example: console.log(core.id)

core.key

• Signature: core.key
• Parameters:
  • none.
• Returns: the public key Buffer for the core.
• Example: console.log(core.key)

core.keyPair

• Signature: core.keyPair
• Parameters:
  • none.
• Returns: { publicKey, secretKey } for writable cores after ready.
• Example: console.log(core.keyPair?.publicKey)

core.discoveryKey

• Signature: core.discoveryKey
• Parameters:
  • none.
• Returns: a derived key used to discover peers without revealing the read capability.
• Example: swarm.join(core.discoveryKey)

core.length

• Signature: core.length
• Parameters:
  • none.
• Returns: the number of blocks currently available on the core.
• Example: console.log(core.length)

core.signedLength

• Signature: core.signedLength
• Parameters:
  • none.
• Returns: the number of blocks signed by a quorum. For single-signer cores, this equals core.length.
• Example: console.log(core.signedLength)

core.contiguousLength

• Signature: core.contiguousLength
• Parameters:
  • none.
• Returns: how many blocks are available contiguously from index 0.
• Example: console.log(core.contiguousLength)

core.remoteContiguousLength

• Signature: core.remoteContiguousLength
• Parameters:
  • none.
• Returns: the largest contiguous length reported by any known remote.
• Example: console.log(core.remoteContiguousLength)

core.fork

• Signature: core.fork
• Parameters:
  • none.
• Returns: the current fork id.
• Example: console.log(core.fork)

core.padding

• Signature: core.padding
• Parameters:
  • none.
• Returns: the per-block padding value, usually 0 unless block encryption is enabled.
• Example: console.log(core.padding)

core.peers

• Signature: core.peers
• Parameters:
  • none.
• Returns: the array of peers currently replicating with this core.
• Example: console.log(core.peers.length)

Events

core.on('close')

• Signature: core.on('close', listener)
• Parameters:
  • listener: callback fired once the core has fully closed.
• Returns: registers a close listener on the core.
• Example: core.on('close', () => console.log('closed'))

core.on('ready')

• Signature: core.on('ready', listener)
• Parameters:
  • listener: callback fired after the core opens its internal state.
• Returns: registers a ready listener on the core.
• Example: core.on('ready', () => console.log(core.key))

core.on('append')

• Signature: core.on('append', listener)
• Parameters:
  • listener: callback fired when local or remote appends change length or byteLength.
• Returns: registers an append listener on the core.
• Example: core.on('append', () => console.log(core.length))

core.on('truncate', ancestors, forkId)

• Signature: core.on('truncate', listener)
• Parameters:
  • listener: callback that receives ancestors and forkId when the core is truncated.
• Returns: registers a truncate listener on the core.
• Example: core.on('truncate', (ancestors, forkId) => console.log(ancestors, forkId))

core.on('peer-add')

• Signature: core.on('peer-add', listener)
• Parameters:
  • listener: callback fired when a peer connection is established.
• Returns: registers a peer-add listener on the core.
• Example: core.on('peer-add', () => console.log(core.peers.length))

core.on('peer-remove')

• Signature: core.on('peer-remove', listener)
• Parameters:
  • listener: callback fired when a peer connection closes.
• Returns: registers a peer-remove listener on the core.
• Example: core.on('peer-remove', () => console.log(core.peers.length))

core.on('upload', index, byteLength, peer)

• Signature: core.on('upload', listener)
• Parameters:
  • listener: callback receiving index, byteLength, and peer for each uploaded block.
• Returns: registers an upload listener on the core.
• Example: core.on('upload', (index) => console.log('uploaded', index))

core.on('download', index, byteLength, peer)

• Signature: core.on('download', listener)
• Parameters:
  • listener: callback receiving index, byteLength, and peer for each downloaded block.
• Returns: registers a download listener on the core.
• Example: core.on('download', (index) => console.log('downloaded', index))

core.on('remote-contiguous-length', length)

• Signature: core.on('remote-contiguous-length', listener)
• Parameters:
  • listener: callback receiving the updated remote contiguous length.
• Returns: registers a listener for core.remoteContiguousLength changes.
• Example: core.on('remote-contiguous-length', (length) => console.log(length))

Static Helpers

Hypercore.MAX_SUGGESTED_BLOCK_SIZE

• Signature: Hypercore.MAX_SUGGESTED_BLOCK_SIZE
• Parameters:
  • none.
• Returns: the recommended 15 MB maximum block size for smooth replication.
• Example: if (buf.length > Hypercore.MAX_SUGGESTED_BLOCK_SIZE) throw new Error('split the block')

Hypercore.key(manifest, options = {})

• Signature: Hypercore.key(manifest, options = {})
• Parameters:
  • manifest: a manifest object or a single signer's public key.
  • options: optional derivation flags:
    • compat: treat a single signer public key as the core key.
    • version: manifest version for single-signer input.
    • namespace: signer namespace for single-signer input.
• Returns: the Hypercore key for the supplied manifest.
• Example: const key = Hypercore.key(manifest)

Hypercore.discoveryKey(key)

• Signature: Hypercore.discoveryKey(key)
• Parameters:
  • key: Hypercore public key.
• Returns: the discovery key derived from key.
• Example: const discoveryKey = Hypercore.discoveryKey(key)

Hypercore.blockEncryptionKey(key, encryptionKey)

• Signature: Hypercore.blockEncryptionKey(key, encryptionKey)
• Parameters:
  • key: Hypercore public key.
  • encryptionKey: base encryption key material.
• Returns: a block-encryption key derived from the two inputs.
• Example: const blockKey = Hypercore.blockEncryptionKey(key, encryptionKey)

Hypercore.getProtocolMuxer(stream)

• Signature: Hypercore.getProtocolMuxer(stream)
• Parameters:
  • stream: a Hypercore protocol stream.
• Returns: the attached Protomux instance.
• Example: const mux = Hypercore.getProtocolMuxer(stream)

Hypercore.createCore(storage, opts)

• Signature: Hypercore.createCore(storage, opts)
• Parameters:
  • storage: path or storage instance for the internal core.
  • opts: internal core options.
• Returns: the internal core object without wrapping it in a full Hypercore instance.
• Example: const internal = Hypercore.createCore('./data', {})

Hypercore.createProtocolStream(isInitiator, opts = {})

• Signature: Hypercore.createProtocolStream(isInitiator, opts = {})
• Parameters:
  • isInitiator: a boolean initiator flag, a framed stream, or an existing Protomux.
  • opts: protocol options, including ondiscoverykey for Corestore-style key management.
• Returns: an encrypted Noise protocol stream with Protomux attached.
• Example: const stream = Hypercore.createProtocolStream(true)

Hypercore.defaultStorage(storage, opts = {})

• Signature: Hypercore.defaultStorage(storage, opts = {})
• Parameters:
  • storage: path or existing hypercore-storage instance.
  • opts: optional storage configuration passed to the default storage implementation.
• Returns: the existing storage instance or a new default Hypercore storage wrapper.
• Example: const storage = Hypercore.defaultStorage('./data')

See also

• Replicate and persist with Hypercore — pair a single core with peer discovery and replication.
• Work with many Hypercores using Corestore — recommended pattern when one process manages multiple cores.
• Corestore — helper reference for creating, naming, and co-replicating groups of Hypercores.
• Hyperbee — sorted key/value B-tree built directly on top of a Hypercore.
• Hyperdrive — filesystem abstraction whose metadata and blob stores are Hypercore-backed.
• Compact encoding — binary encoding toolkit used to structure Hypercore block payloads.