If you're using gedit to edit existing files, the default preference is to "Create a backup of files before saving". If you would like to change this default behavior, open gedit, then go up to the Menu bar and select Edit --> Preferences. Click on the Editor Tab, and Uncheck the box under File Saving in order to turn it off.

Another common ~ file creator is emacs - some would argue it's even more popular/common than gedit.