You shouldn't need to install a Git server on your local machine to work with Git. Git is a distributed version control system (DVCS), ss such there really is no idea of a Client and Server. Instead you just have clients, as many clients as you want, which can talk to each other.

So when you type git init in a directory and create a git repository, you've got all you'll need to commit, merge, branch, and ultimately use git. For all intensive purposes, this is the master source, the "god" source, equivalent to the SVN server. Unlike in subversion, where there is a centralized source, only one true "god"/master copy, Git works in a way that everyone who clones a git repository has their own "god"/master copy. So when you share code, each person gets the theoretical equivalent of an SVN server on their machine in the form of a portable git repository. This strongly differs from the centralized model (and lends to it's distributed nature) where if your source becomes deleted it does not affect anyone else. In fact if you lose your source you can clone from someone else who has a copy of the repository and continue as if nothing happened. So long as N+1 people have a copy of the code, if all of N lose it the code will live on.

What most DVCS tools provide is a flexibility of workflow. Many people adopt a centralized workflow using services like GitHub or Bitbucket to host a remote copy of their repository. In this setup there is still fundamentally just two git repositories. One on your machine and one on the services remote servers. I strongly recommend reading the free git-scm book before getting started to familiarize yourself with what git is, how it works, and how to use it.