We Run iSCSI over the Internet

· systems · Source ↗

TLDR

  • scsipub serves iSCSI targets to public internet clients using Elixir/BEAM, Ranch listeners, COW overlays, and Caddy-terminated TLS on port 3261.

Key Takeaways

  • One BEAM process per Ranch-accepted TCP connection maps cleanly to one iSCSI session; ~2.5 KB heap per process scales to tens of thousands without thread-pool tuning.
  • COW overlays use sparse files plus a per-sector bitmap: base image stays read-only, writes cost only actual sectors touched, and non-persistent tiers delete overlays on TCP close.
  • TLS cert rotation is handled by inotifywait copying Caddy’s ACME cert to a shared path; sni_fun re-reads PEM per handshake with a 60-second cooldown and mtime guard to avoid syscall overhead.
  • open-iscsi quirks that each cost a day: IQN slashes become filesystem path separators breaking node creation; SendTargets must advertise a reachable address not sockname(); static -o new login required to avoid portal-override from discovery responses.
  • Multi-LUN sessions and SCSI-3 Persistent Reservations are implemented for real cluster software compatibility; LUN routing dispatches by pdu.lun into a per-session backend map.

Hacker News Comment Review

  • Discussion is thin but positive; the builder himself is present and confirmed the free tier requires no signup, with iscsiadm discovery against scsipub.com landing a 64 MB scratch disk immediately.
  • No technical disagreement or identified risks surfaced; comments treat the open-iscsi IQN slash edge case as the standout practical finding.

Notable Comments

  • @qdotme: confirms free-tier demo: discovery + login to iqn.2025-01.pub.scsipub:blank gives a live block device with no account required.

Original | Discuss on HN