Which team do you belong to? Mono-repo or Multi-repo?
In case you are from Multi-repo, or you are thinking about it, explore WEST.
In case you don’t know what this article is about… you are lucky! May be your software stack has not exploded yet to parallel development; Thinking about multiple repositories is not a question in your mind yet.
Multi Repo
Some times a single Git repository is not enough for your code base. You split them logically it into multiple different repositories. What is logical is up to your product. The topic of how you split your code base in into multiple repositories would take a dedicated discussion. But for example, may be you choose to split into separate repository for documentation, portable API Layer, core library, non portable hardware specific code, tests, etc.
Here are some of the reasons for splitting your code into multiple repositories.
Access and Authority Control
-
E.g. API layer would need a stringent scrutiny for change, while tests and documentation would see relatively relaxed attention.
-
The set of engineers approving any change to core library would be different from those approving changes to tests.
-
Any change in library would need to pass through some level of testing before being approved for mainline.
Reuse
-
Same sub repository can be re-used across various different projects and teams.
-
In some cases, the repository itself is external to you and you are using it as-is.
-
Instead of manually copying files from one location to other, collaboration/re-use can be leveraged directly by Git.
If you have done this many times, you will know what are the pitfalls of such manual copy.
WEST
WEST is a Python based project / tool / library / utility. It can be used for managing multiple git repositories (and more).
Extensive documentation for west
is available as a part of the
Zephyr
Project
How to set it up?
Generally, it starts with creating a west.yml
in your top level
repo. Something like this:
1---
2manifest:
3 # The version of this manifest
4 version: '1.2'
5 # Default project attributes
6 defaults:
7 remote: c6h6-demo
8 # Short names for project URLs
9 remotes:
10 - name: c6h6-demo
11 url-base: git@github.com:c6h6-demo
12 # A list of projects managed by west
13 projects:
14 - name: sub-r1
15 description: West Demo sub repo r1
16 repo-path: west-sub-r1
17 revision: main
18 self:
19 path: _top
You want to try?
Do you want to see it / use it for yourself? There’s a demo site for you to play.
Install west
1pip install --user west
Initialize a repo
Run the command:
1west init --manifest-url git@github.com:c6h6-demo/west-top.git
The output would be:
1=== Initializing in /Users/c6h6/data/p/west-demo/try2
2--- Cloning manifest repository from git@github.com:c6h6-demo/west-top.git
3Cloning into '/Users/c6h6/data/p/west-demo/try2/.west/manifest-tmp'...
4--- setting manifest.path to _top
5=== Initialized. Now run "west update" inside /Users/c6h6/data/p/west-demo/try2.
The tree would look like:
1.
2└── _top
3 ├── README.md
4 └── west.yml
As you can see, it contains the west.yml
and README.md
. But it
still does not contain the dependent repositories.
Bring the dependent sub repositories
Run the command:
1west update --keep-descendants
The output would be:
1=== updating sub-r1 (sub-r1):
2--- sub-r1: initializing
3Initialized empty Git repository in /Users/c6h6/data/p/west-demo/try2/sub-r1/.git/
4--- sub-r1: fetching, need revision main
5From github.com:c6h6-demo/west-sub-r1
6 * branch main -> FETCH_HEAD
7HEAD is now at 36fc942 Initial commit
8HEAD is now at 36fc942 Initial commit
The tree would look like:
1.
2├── _top
3│ ├── README.md
4│ └── west.yml
5└── sub-r1
6 └── README.md
Now, we have sub-r1
and it’s contents.
Check for changes
Run the command:
1west status
The output would be empty.
But, if we modify sub-r1/README.md
, the output would be:
1=== status of sub-r1 (sub-r1):
2HEAD detached at refs/heads/manifest-rev
3Changes not staged for commit:
4 (use "git add <file>..." to update what will be committed)
5 (use "git restore <file>..." to discard changes in working directory)
6 modified: README.md
7
8no changes added to commit (use "git add" and/or "git commit -a")
For little silent output:
1west status -s
The output would be:
1=== status of sub-r1 (sub-r1):
2 M README.md
Verbose status check
For no changes, if we run the command:
1west -v status
The output would be:
1=== status of manifest (_top):
2On branch main
3Your branch is up to date with 'origin/main'.
4
5nothing to commit, working tree clean
6=== status of sub-r1 (sub-r1):
7HEAD detached at refs/heads/manifest-rev
8nothing to commit, working tree clean
west and manifest-rev
Run the command:
1git -C sub-r1 status
The output would be:
1HEAD detached at refs/heads/manifest-rev
2nothing to commit, working tree clean
As you can see, west
has created a local reference commit of
manifest-rev
to refer to the revision as selected by the manifest.
Other commands
If you want to run some command in all git repositories, run the command similar to:
1west forall -c "git fetch"
The output for above command would be:
1=== running "git fetch" in manifest (_top):
2=== running "git fetch" in sub-r1 (sub-r1):
3From github.com:c6h6-demo/west-sub-r1
4 * [new branch] main -> c6h6-demo/main
Switch away from manifest-rev
As you can see with west output of west -v status
, the working tree
refers to manifest-rev
. If you want to switch to top of the branch
you can take help of the following command:
1west list -f "git -C {path} checkout {revision}" | grep -v " HEAD"
It uses the built in command west list
to generate a command to
check out directly the branch for that repo.
As of now the output would be only:
1git -C sub-r1 checkout main
But if you have more repositories, the output would change.
How does it compare to git repo?
Git Repo is a tool that is extensively used for the Android project.
Key difference between both the tool is that repo
is xml based,
while west
is yaml based. But both the tools try to solve a similar
problem.
Read more at WEST document.
How does it compare to git submodules?
Well, in simple words, west
and repo
exists because git
submodules
is not enough for decent sized work flows.
Read more at WEST document
What could be better?
west
is a nice tool, and I think for quite some time I will stick to
it, but here are few things that I feel west
potentially can do
better after some initial run.
-
As compared to git-repo, here are few improvements that I feel
west
could do better.-
repo
does not use a default upstream repository, neither shouldwest
-
i.e.
repo init
will fail. But,west init
will clone Zephyr.If
west
wants to be true generic / general project, it should not be coupled withZephyr
-
-
repo
can work in parallel, it would be nice ifwest
could too.
-
-
Shell Completion
Shell completion as reported in official documentation, using
west completion
, which helps for completion using bash/zsh/fish, is part ofzephyr
repo. Having this outside ofzephyr
would help users.
Comments