You've got this awesome project you built internally. It started small, maybe lived in a giant monorepo, or perhaps its own repo that also accumulated some... internal peculiarities. Now, the time has come: you want to share it with the world! Open source it! 🎉
That initial excitement often hits a speed bump: how do you actually get just that project out without exposing the entire monorepo, confidential history, accidental secrets, or internal configurations? And maybe even trickier: once it's out, how do you keep the public version updated if development still primarily happens internally?
I've wrestled with this myself. Manually copying files is error-prone. git filter-repo (or its predecessor filter-branch) is powerful but feels like handling dynamite – one wrong move and you've either leaked history or permanently mangled the public repo. And keeping things synced later? Often involves complex scripts, manual cherry-picking marathons, or just giving up and letting the public version stagnate.
Wouldn't it be nice to have a tool that helps manage this process? Something that automates the tedious parts but keeps you firmly in control for the critical review steps?
That's why I built oss-porter.
What is oss-porter?
oss-porter is a command-line tool written in Rust, designed specifically for this internal-to-public workflow. Its philosophy is:
- Configuration is Key: Define what needs to be ported and how in a simple
~/.oss-porter.tomlfile. Manage multiple projects easily. - Safety First: Provide mechanisms for clean extraction and emphasize manual review. Help prevent accidental leaks.
- Sustainable Updates: Offer a guided, interactive workflow to sync ongoing internal development to the public repository after the initial release.
How Does It Work? (The Core Ideas)
Instead of just copying files or blindly filtering history, oss-porter uses a more structured approach based on your configuration:
Configuration (
oss-porter config ...): You start by tellingoss-porterabout your project(s) using commands likeconfig initandconfig add(or by editing~/.oss-porter.toml). You define things like:internal_repo_path: Where the private source lives.project_subdir: Which specific subdirectory within that repo is the project you care about.output_path: A separate, dedicated directory where the clean, public version will be managed locally.history_mode:"clean_slate"(default, copies files, no history) or"preserve"(usesgit-filter-repo, requires manual vigilance!).internal_branch/public_branch: Which branches to track and push to (defaults tomain).public_repo_url: The URL for the final GitHub/GitLab destination.
Initial Extraction (
oss-porter extract <id>): Based on the config, this command creates theoutput_path, populates it using the chosenhistory_mode, initializes a fresh Git repo (if clean slate), adds basics like.gitignoreand a placeholderLICENSE(if configured), and crucially excludes the internal state file. This step requires heavy manual review of theoutput_pathafterwards!Ongoing Updates (
oss-porter update <id>): This is where it gets interesting for maintainers. It doesn't just re-copy. Instead:- It reads a state file (
.oss_porter_state.toml, committed inside your internal repo's project subdir) to know the last internal commit that was synced. - It finds new commits on the configured
internal_branchthat touched theproject_subdir. - It then interactively walks you through each commit:
- Shows you the
git diff(relative to the project subdir). - Asks: Apply this commit?
[Y]es / [n]o / [s]kip / [A]ll remaining / [q]uit
- Shows you the
- It applies the approved commits as patches (
git am) to youroutput_pathrepository. - If conflicts occur, it pauses and tells you how to resolve them manually.
- When done, it updates the state file and prompts you to commit that state change back to your internal repo.
- It reads a state file (
Checks & Push (
oss-porter check <id>,oss-porter push <id>): Helper commands to run basic sanity checks (path deps, license) on theoutput_pathrepo and push it to the configuredpublic_repo_url.
Use Cases (Especially for Corporate OSS)
- Releasing a Library from a Monorepo: Define the library's subdir, use
extract clean_slate, manually review theoutput_paththoroughly, set up the internal state file,push, and then useupdateto sync future internal fixes/features. - Open Sourcing an Existing Internal Tool: If the tool had its own repo but with internal cruft,
extract clean_slateprovides a fresh start. If history is really valuable and you are confident you can clean it,extract preserveis an option (review required!). Useupdatefor ongoing sync. - Maintaining Multiple OSS Projects: Define all your managed projects in the central
~/.oss-porter.tomland useoss-porter update <project-id>for each as needed. The shared state file (committed internally per project) ensures collaborators can pick up where others left off. - Controlled Syncing: The interactive
updateallows you to explicitly skip internal commits that shouldn't go public (e.g., internal refactoring, commits accidentally touching the subdir, temporary hacks) without complex Git gymnastics.
Example Quick Flow
# 1. One-time setup
oss-porter config init
oss-porter config add # Follow interactive prompts
# 2. Initial Extraction (creates /path/to/output defined in config)
oss-porter extract my-project
# 3. !!! CRITICAL MANUAL REVIEW in /path/to/output !!!
# (Clean secrets, fix deps, add LICENSE/README, git commit fixes)
# 4. Create & Commit State File internally
# (Create .oss_porter_state.toml in internal repo's subdir with extract hash)
# (git add .oss_porter_state.toml && git commit ...)
# 5. First Push
oss-porter push my-project
# --- Later ---
# 6. Make changes internally, commit & push internally
# 7. Sync updates interactively
oss-porter update my-project # Review diffs, select Y/n/s/A/q
# 8. If prompted, confirm commit of updated state file internally
# 9. !!! CRITICAL MANUAL REVIEW & TEST in /path/to/output !!!
# 10. Push updates
oss-porter push my-project
Safety First!
It's worth repeating: oss-porter helps automate the mechanics, but it does not eliminate the need for careful manual review. Especially after extract and after update (particularly if you use the [A]ll option), you must inspect the code and potentially the history in the output_path directory before pushing publicly.
Try it Out!
This is still a relatively new tool, but it solves a problem I've personally encountered frequently.
- Find the code: https://github.com/normano/oss_porter
- Installation:
cargo install oss-porteror build from source.
I'd love to hear feedback, suggestions, or contributions if this is a problem you face too! Let me know what you think.