Method (I): Using Firefox's Preferences menu: (preferred)

A. From Firefox's global menu, select Edit -> Preferences (or press Alt + E followed by N) to open Firefox Preferences.

(Snapshot taken in Ubuntu 12.04)

B. Select the Applications menu and search for BitTorrent seed file or application/x-bittorrent or torrent. Change the Always ask or Use gedit (default) to transmission-gtk by selecting Use other... and click File System in the side bar, then browse to /usr/bin.

C. In /usr/bin search for transmission-gtk and double click to open.

D. That's it! Now the .torrent files will be opened with transmission.

Method (2): Editing the mimeTypes.rdf file:

Step 1:

Open the mimeTypes.rdf file with text editor of your choice:

gedit ~/.mozilla/firefox/ythemo16.default/mimeTypes.rdf &

(Save a backup of this file.) Search for the container:

<RDF:Seq RDF:about="urn:mimetypes:root">

 ...

 ...



</RDF:Seq>

and insert this:

<RDF:li RDF:resource="urn:mimetype:application/x-bittorrent"/>

so that, now the container looks something like this:

<RDF:Seq RDF:about="urn:mimetypes:root">

 ...

 ...



 <RDF:li RDF:resource="urn:mimetype:application/x-bittorrent"/>

</RDF:Seq>

(step 1 snapshot)

Step 2:

Now, within the container:

<RDF:RDF ... ...>

 ...

 ...

 

 ...

 ...



</RDF:RDF>

insert these:

<RDF:Description RDF:about="urn:mimetype:handler:application/x-bittorrent"

               NC:alwaysAsk="false"

               NC:saveToDisk="false">

 <NC:possibleApplication RDF:resource="urn:handler:local:/usr/bin/transmission-gtk"/>

 <NC:externalApplication RDF:resource="urn:mimetype:externalApplication:application/x-bittorrent"/>

</RDF:Description>

<RDF:Description RDF:about="urn:mimetype:externalApplication:application/x-bittorrent"

               NC:prettyName="transmission-gtk"

               NC:path="/usr/bin/transmission-gtk" />

<RDF:Description RDF:about="urn:handler:local:/usr/bin/transmission-gtk"

               NC:prettyName="transmission-gtk"

               NC:path="/usr/bin/transmission-gtk" />



<RDF:Description RDF:about="urn:mimetype:application/x-bittorrent"

               NC:fileExtensions="torrent"

               NC:description="BitTorrent seed file"

               NC:value="application/x-bittorrent"

               NC:editable="true">

    <NC:handlerProp RDF:resource="urn:mimetype:handler:application/x-bittorrent"/>

</RDF:Description>

so that the container will look something like this:

<RDF:RDF ... ...>

 ...

 ...

 

 ...

 ...



 <RDF:Description RDF:about="urn:mimetype:handler:application/x-bittorrent"

               NC:alwaysAsk="false"

               NC:saveToDisk="false">

  <NC:possibleApplication RDF:resource="urn:handler:local:/usr/bin/transmission-gtk"/>

  <NC:externalApplication RDF:resource="urn:mimetype:externalApplication:application/x-bittorrent"/>

 </RDF:Description>

 <RDF:Description RDF:about="urn:mimetype:externalApplication:application/x-bittorrent"

               NC:prettyName="transmission-gtk"

               NC:path="/usr/bin/transmission-gtk" />

 <RDF:Description RDF:about="urn:handler:local:/usr/bin/transmission-gtk"

               NC:prettyName="transmission-gtk"

               NC:path="/usr/bin/transmission-gtk" />



 <RDF:Description RDF:about="urn:mimetype:application/x-bittorrent"

               NC:fileExtensions="torrent"

               NC:description="BitTorrent seed file"

               NC:value="application/x-bittorrent"

               NC:editable="true">

    <NC:handlerProp RDF:resource="urn:mimetype:handler:application/x-bittorrent"/>

 </RDF:Description>

</RDF:RDF>

(step 2 snapshot)

Step 3:

Save and close the file. Then restart firefox.