How to deal with Git and Gitlab for Gambas

A repository is like a file system that tracks modifications over the time.

  • Git is the repository software that manages the Gambas source code.

  • Gitlab its the interface software that manages the Gambas source work.

Git basics and facts

The git are based on two kind of principles: the timeline and the commits, all the changes are references in time, and those references are "commits" in the "timeline".

Git it's de-centralized, a local working copy and a remote reference to central working main.

  • In local, can be made many "reference changes" until "that changes are upload" to the central. In the main central all of them will be "reference changes merged".

  • In main, all the "reference changes" are "merged", the git software takes cares of "conflicts" before the "upload", in local copy at the side of each user.

Performances workflow

The local copy its performed by each user or developer. The central work its commanded by the gitlab software using a git daemon.

  • The developers or users: must take care of "fetch"s to make "pull"s updates to their code, and then "push" to "remote" central work

  • The administrator or member: must take care of "merge request"s to the base code, so then all the others can then "pull" to their local copys

Setup and install git

Linux MACOS W32
apt-get install git
yum install git
depends of each distro 		http://git-scm.com/download/mac http://msysgit.github.io/

Using git in short way

This it's a quick reference, if you are new to concepts of git or will to code at, strongly recommend read the section of this document Gambas git reference. All is done with the git command.

  • git clone makes a copy of the repository on your hard disk, and adds a only hidden .git directories in only one place, here are all your credentials.

  • git add --all takes one file/dir changed and mark to commit before upload to the central repository

  • git push upload all the "add"ed and "commit"ed changes to the central repository.

  • git fetch download recent changes but does not alter your local changes, that's the power of git, de-centraliced

  • git pull takes thats changes downloaded and updated your local copy of the repository, if there's not conflicts.

Some quick example cases; If you are not familiar with git workflow,or what to code in Gambas, read at the section Gambas git reference..

  • 1. Making local copy:

git clone https://gitlab.com/gambas/gambas.git gambasdevel
cd gambasdevel

  • 2. Updates from upstream:

git fetch
git pull

  • 3. See changes and logs, both:

git whatchanged

  • 4. Dont retrieve all, only last 3 commits:

git clone --depth=3 https://gitlab.com/gambas/gambas.git gambasdevel
cd gambasdevel

  • 4.1. Retrieve missing pieces from that 3 commits:

git fetch --unshallow

  • 5. Clone a only lasted stable:

git clone --depth=3 https://gitlab.com/gambas/gambas.git gambasdevel
cd gambasdevel && git checkout $(git tag | sort -V | tail -n 1)

  • 6. Clone only one branch (ex, stable):

git clone -b stable --single-branch https://gitlab.com/gambas/gambas.git gambasstable
cd gambasstable

  • 6.1. Retrieve missing branches from single branch:

git config remote.origin.fetch "+refs/heads/*:refs/remotes/origin/*"
git fetch origin

Branches are used to develop features isolated from each other and continuous parallel work. The master branch is the "development" branch and there's also the "stable" branch.

Gambas Development organization

Gambas source code it's hosted on https://gitlab.com/gambas/gambas

  • The first gambas is the name of the group of GitLab users at development.

  • The second gambas is the name of the project.

The Issue tracking and wiki documentation are outside of Gitlab services, managed by Gambas technologies at http://gambaswiki.org/

Gambas git organization

In git, the repository of Gambas are virtual file systems based on the "commits" references through a "timeline", the artifacts of that are explained quickly here::
commits Are references to changes in the repository, commits are kind of like ref-counted, it means a change regard the previous in time.
Are made by each developer at local copy and then push in the central repository.
A commit object contains three things:
  • A set of files, reflecting the state of a project at a given point in time.

  • A parent commit, the reference to the previous state of that files.

  • An SHA1 name, a 40-character string that uniquely identifies the commit object.

branches Are parallel virtual filesystem/commits, regard of the normal timeline repository. Are create from commit references.
Any new feature from any developer has a parallel branch to work until are merged into main development.
Gambas repository alway have two fixed branches:
  • master : are the developer branch, it version its the "rolling" unstable development version of Gambas, which is all the snapshots of Gambas that happen between last stable release and the next to release.

  • stable branch that points only to lasted released.

tags Is just a named reference to a commit. Git uses two main types of tags:
  • lightweight a branch that doesn’t change, just a pointer to a specific commit in timeline.

  • annotated stored as full real objects; they’re checksummed, GPG signed, with name, email, date and message.

Gambas repository only has lightweight due Gitlab use. If where Github, was'ed annotated.
  • All the stable releases are tags, all the Gambas 1.X series, Gambas2 2.X series and lasted.

Gambas gitlab organization

Unlike other Git Content Management interfaces, in Gitlab each Project its linked to its git repository. So The Gambas git repository it's also the Gambas Gitlab project.

The Gambas Gitlab organization is composed by three artifacts:
  • The Gambas namespace: is the Gambas unique name used also as the group name, over all the Gitlab community ---> https://gitlab.com/gambas/gambas

  • The Gambas repository: is the Gambas reference of the git repository source code, under the group name ----> https://gitlab.com/gambas/gambas

  • The Gambas contributions: is the way of how external people send code pieces or artifacts ----> https://gitlab.com/groups/gambas/merge_requests

Remenber that Issue tracking and wiki documentation are outside of the Gitlab services, Gambas uses their own software for wiki and bugtraking.

Git workflow and bifurcations

All people, to contribute need made usage of Gitlab workflow and the name spaces, in Gitlab there a easy way to make a fork: in main Gambas project Gitlab page, click on the "Fork" button or consult the Gitlab fork guide at https://gitlab.com/gambas/gambas/forks/new, here's the overall process:

  1. Create a GitLab account and the username will be your name-space, take care of disabling any blocking software.

  2. Fork to your namespace username the git code, go to the Gambas project page, and click the "Fork" button.

  3. In the Gitlab interface go to the file to change, when clicked, Gitlab automatically will manage a branch for that change, and will open an editor to make changes

  4. One changes are save with "Save" button, Gitlab interface automatically offers a interface to make a "merge pull request"

  5. If: the commit message, the code changes and the pull message are made in right way with right format, will be reviewed and merged by a Gambas member.

For more advanced contribution please refer to the contribute guide.

Git and the local copy

The most important from developer view, are the git local copy repository: consists of three "trees" maintained by git:

  • the local copy it's the Working Directory which holds the actual files.

  • the Index which acts as a staging area.

  • the HEAD which points to the last commit made respect the local clone.

Each time anything is changed in the local copy repository, theres a commit reference always regarding the previous files, these references are SHA-1 strings, very large, the gitlab interface easily permits to only use 6 to 8 digits to make human readable marks. Those references are produced in "commits", in "push" uploads and in "merge" commits or when sync local copies of the repository.

Git Clone and Checkout: a local copy

A local copy its a git clone from the desired repository, in the local user file system from remote main/copy repository: Nearly Every Operation Is Local. Most operations in Git only need local files and resources, no information is needed from another computer on your network.

  • In case if: clone a local copy from main repository only to use, and no uploads to upstream remote will be made:

git clone https://gitlab.com/gambas/gambas.git

  • In case if: clone a local copy from forked repository to use and then later made uploads to upstream as merge requests:

git clone https://gitlab.com/myusername/gambas.git

Once with the local copy repository, will have a file system with virtual others by "references" from "commits, the "trees" maintained by git.

  1. There's your Working Directory which holds the actual files.

  2. There's the Index which acts as a staging area

  3. There's the HEAD which points to the last commit.

In most operations have a network dependency, but Git have it simply directly from your local database. Because you have the entire history of the project right there on your local disk, most operations seem almost instantaneous and network-less.

Git Commits and Push: commits are not uploads!

Unlike other VCS, major difference is about it's data, more like a set of snapshots of a miniature file system, like a stream of snapshots.

In Subversion, a commit implicates a upload, in Git its not a upload, a commit its only just a link to the previous identical file it has already stored and the diff to represent.

How that works: So the commit in git are disaggregated in more parts, first, like subversion, added what are changed, then marks all that diff changes in the commit message, and lasted take all these as a packet and made the upload to upstream.

First, the add sub command added the files or directories changed:

git add <filename>

If you want to unconditionally save all changes and removals, just:

git add --all *

Be carefully, the wildcard it's not same as . and can be considered a directory a wildcard recursively of all it's content.

Then the commit mark it's a couple of changes as reference mark, means that there is very little you can’t do if you’re offline or off VPN. If you get on an airplane or a train and want to do a little work, you can commit happily until you get to a network connection to upload.

git commit -m "Commit message, a couple of changes with one commit, but not a upload yet"

Every time you commit, or save the state of your project in Git, it basically takes a picture of what all your files look like at that moment and stores a reference to that snapshot. Now the file is committed to the HEAD, but not in remote/upstream repository yet, means not uploaded but marked modified and committed locally.

The push sub command it's to send those changes to the remote repository if you are a Gambas contributor or to your forked repository if your are a external contributor:

git push

All the commits and change always be add it to the Index and with branch you can manage where to save the references of commits

The push will always ask for your credentials, your username and password of Gitlab account.
If the git repository was cloned with ssh event https, you will need a GPG key pair.

Git Branches and flavors: for code develop

Branches are used to develop features isolated from each other. The master branch is the "development" branch.

Use branches for develop new specific artifacts, components or features, and then when complete can be merged into develop or "master" by a "merge request" behaviour.

Change master to whatever branch you want to push your changes to or create a new branch, with same command:

git branch gb.linux.kernelxploit

Now your are in the incredible "gb.linux.kernelxploit" branch to develop a great exploit new component to reuse in kali linux or VenenuX or, well, lest continue. If the branch exist, you will switch to, if not, it will be created and will be committed to upstream when do the push command.

If the brand already exist, you will are working in it, if not will create a new branch, please follow guidelines, branch its only to develop features not ready to maintain in main develop

Take in consideration that a branch is not available to others unless you push the branch to the remote/central repository in gitlab, and those made a fetch & pull, see more in this guide

Then commit changes and later upload with push command.

In Gambas git repository the work are in the branch, master are the principal and the develop working, there's a stable branch and some other of not yet working and in develop work like the cmake future feature.

To use, correct or delete any branch switch back to master and then

$ git checkout master

..and delete the branch again

$ git branch -d gb.linux.kernelxploit

..or maybe switch to stable:

$ git branch stable

Git tags and checkouts: get some point specific

Cut out the middleman (gitlab). To make a new tag, use

$ git tag v3.11.0 <3.11.0-establishing commit>

When the <3.11.0-establishing commit> its a SHA1 commit part mark in the history of the repository changes, and remenber to have to push to gitlab then to make it available to others, due all are happened in your local repository.

Git archive: Compressing and distribute with the tags

Making an archive is as you would hope, by example making for the 3.11.0 release will be as:

$ git archive --format tar v3.11.0 | bzip2 >gambas-3.11.0.tar.bz2

The archive will contain the snapshot of the source tree at the v3.11.0 tag, i.e. the commit you gave above. It won't include any git information, e.g. I get a nice 33 MiB archive out of using the above with v3.9.0. Symbolic links will be preserved.

Git local updates: fetching, pulling and pushing

Due git are de-centralised, local updates can be made in two cases: from origin upstream or from personal upstream namespaces. Normally that are automatically with the following command:

$ git fetch
$ git pull

The fetch will retrieve only the lasted but not will override your local copy files.. so in that point you can made some diff compares before pull the changes.

But if you want the changes from main Gambas git repository, need perform some operations. If are at Gitlab can made graphically, here we will explain how to do for the local copy:

Setup local Git copy by adding remote Gambas repository as a second remote :

$ git remote add upstream https://gitlab.com/gambas/gambas.git

Now can normally fetch updates to first makes diff's:

$ git fetch upstream master

Or also directly pull changes event first fetching:

$ git pull upstream master

This command will take the changes from the master branch of the upstream remote.

If you made commits to your version of the repository, it will merge them with the new changes, creating a new "merge commit". The git sub-command will as to a "merge sync message" before that operation if there many more "commits to update" in your local copy.

When the merge is complete, you can simply use git push to push all these changes to your version of the repository.

If you have any Merge Request pending, (recommended in a separate branch) they will get updated automatically (of course if your pushed was in that branch).

Git changes and state: what are changed detailed

Git can offers status of pending "non-marked" changes.. git "status" sub-command got what are uncommitted and what are committed to upload:

git status

With the "diff" sub-command you can see the differences before made a "commit" :

git diff

To see all the history log of activity, the normal and already know ed git log sub-command are so limited, theres a more detailed command:

git whatchanged

This only works with current cloned history, so if you cloned git clone --depth=1 will only show one commit due depth has only one level.

Gambas: developers and/or contributors

There's two kind of contributors of Gambas coding, the core developers know ed as Gambas Members at Gitlab, and the contributor developers that need made usage of Gitlab workflow.

To make it a concurrent core developer as a Gambas Member, need in same order those requisites:

  1. Ask to get approved at mail development list; see it in /wiki/howto/doc/forum

  2. A user account on Gitlab.com, using https://gitlab.com/users/sign_in

  3. Have a very good knowledge of Git using the Gambas git reference and "Pro Git" https://git-scm.com/book/en/v2

  4. Knowledge in use branches for develop new components or features

  5. Usage of Git commit log and messages format for commits messages as mentioned in sections below

Once one all requisites are fit, then clone a working directory (name it as you like):

$ git clone https://gitlab.com/gambas/gambas.git

And create a working brand to pull in main master branch of your new develop feature:

$ git branch gb.new.feature

If no new feature will be develop by you and want only deal with patches and submit fixes, use pull request way. Please see guide.

Git commit log and messages

As per the Git commit message convention, the first line of each commit is a short description of what it contains.

That's first line it appears in each git head logs, as well as in the GitLab interface. The rest of the message commit lines will be a log commit content, that must be in special format.

The git and Gitlab interface has 3 moments to write messages:
  • commit : here you will write the changes in the needed format

  • merge: this are a message notify update to the fork or from..

  • request: this are only identificative to the web interface

Gambas changelog format in commits

In order to automatically generate changelogs for each release, commits logs in the Gambas repository have to follow a very specific format.

When user made a git commit command, the message comment to write must be in format, the other messages for merge or pull request not.

Then, the commit message consists of the following parts :

  • A slot, between square brackets (e.g. [GB.QT4])

  • One or more modifications, each prefixed with a tag, which is either * NEW: , * BUG: , or * OPT: , with a space at the end.

The slot's name is the one of the component modified (in uppercase), or one of these if the changes do not affect a component :
  • [INTERPRETER] for changes in the interpreter (gbx3).

  • [COMPILER] for changes in the compiler (gbc3).

  • [ARCHIVER] for changes in the archiver (gba3).

  • [INFORMER] for changes in the informer (gbi3).

  • [DEVELOPMENT ENVIRONMENT] for changes in the IDE (gambas3).

  • [CONFIGURATION] for changes in the automake/autoconf configuration process

  • [WIKI CGI SCRIPT] for changes in the wiki CGI script.

  • [WEB SITE MAKER] for changes in the Gambas web site generator.

  • [EXAMPLES] for changes in any example.

The tag's name is one of the following:

  • NEW is for new features or translations, updates or other improvements;

  • BUG is for bug fixes and other corrections

  • OPT is for optimizations

Things without an impact for the user (such as refactoring or code cleanups) should not end up in the Changelog.

All lines without a tag will not appear in the generated changelog, but if you want a modification to span across multiple lines, you will have to prefix it with two spaces. 

This commit contains things, adds stuff and has lots of fluff.


[GB.QT4]
* NEW: Added things to the component.
* BUG: Fixed a bug in the Foo function.
* NEW: Added this very long modification...
  ...and it takes more than one Line To Write it.


[COMPILER]
* BUG: What an awful bug!
* OPT: Make things go faster.

[GB.GTK3]
* NEW: The component is now complete!

Commit mailing-list

There is a mailing-list that gets a mail each time somebody commits a new revision. So this way you always know if you have the latest revision on your hard disk or not.

Gitlab has a service that automatically in each push/upload of commits send a mail to this mailing-list

To subscribe to this mailing-list, go to the mailing-list page on the web site. The name of the mailing-list is gambas-devel-svn (for now).

Caveats

All the git command can be made in any place while are inside the root of the directory working copy.. but obviously must take in consideration the references regards the current working directory.

Conflicts

Somebody can modify a file in the repository, while you have modified the same file on your hard disk

(WIP)

Gambas IDE git integration

Gambas currently only has support inside IDE to SVN commands, git support are on the way. The development are made in the gb.git branch of the repository: https://gitlab.com/gambas/gambas/tree/gb.git

See also