You cannot upgrade a research lab that is current researching something.
You can upgrade other buildings on that planet, or upgrade a Research Plant on another planet. It isn’t very beneficial to have multiple Research Plants though, since you can only have one set of research going at once.
Building a second research plant is really only useful if you plan on destroying your first one. Either to start a new colony, or to reclaim some of the fields.