vncviewer allows to specify the display number in the connection string:

If you run the viewer with no arguments it will prompt you for a VNC
server to connect to. Alternatively, specify the VNC server as an
argument, e.g.: vncviewer snoopy:2

where 'snoopy' is the name of the machine, and '2' is the display
number of the VNC server on that machine. Either the machine name or
display number can be omitted. So for example ":1" means display
number 1 on the same machine, and "snoopy" means "snoopy:0" i.e.
display 0 on machine "snoopy".

Source: manpage for vncviewer

I could not find an explicit option in Remmina to specify display number, however:

By default, a VNC server will listen for connections from a VNC viewer
on TCP port "5900+N", where N is the "Display Number" (usually just
zero). So a VNC server setup for Display-0 will listen on TCP-5900,
Display-1 is TCP-5901, etc.

Additionally, the VNC server will listen for connections from a
web-browser on TCP port "5800+N", where N is as above. If you point a
web-browser to this port, the VNC server will automatically provide a
Java VNC viewer that runs right in your web-browser. This Java Viewer
will then exchange data with the VNC Server on the same ports a normal
VNC Viewer would use: "5900+N".

(source)

So I guess you may try to connect to a specific display by specifying port number in connection settings.