In 1926, the Department of the Navy authorized the establishment of Reserve Officer Training Corps units at six colleges in order to “afford systematic instruction and training to qualify students for appointments as ensigns in the naval reserve upon graduation.” The six schools chosen were Georgia Tech, Harvard, Yale, Northwestern University, University of California at Berkeley, and the University of Washington. On 10 September 1926, Commander J.J. London and his staff began interviewing 175 applicants for the unit, which was set up on North Avenue. Membership was limited to 60 freshmen who were United States Citizens, at least 14 years of age, and mentally and physically qualified for military duty.

The initial courses consisted of training in navigation, gunnery, and seamanship - supplemented by drill - conducted aboard three 24-ft. whaleboats on Piedmont Lake. They also conducted exercises on a simulated bridge of a ship and operated a 4-in. naval gun located at the unit. When a new Naval Armory was completed in 1935, the facilities expanded to include a more elaborate bridge with a complete communications system, an operational ship’s boiler, a lifeboat with davits, a flagbag, and signal lights. Midshipmen from all six units trained together on summer cruises aboard battleships such as the USS Florida, USS New York, USS Arkansas, USS Mississippi, and USS Wyoming.

During the first ten years, membership varied between 200 and 270 midshipmen. As a result of the limited emergency proclaimed on September 8, 1939, a Naval Reserve Indoctrination School was set up, during summer months to provide ten weeks of instruction in naval subjects to newly commissioned ensigns, from colleges across the nation. On July 1, 1943, the V-12 program went into effect to give experienced enlistees the opportunity to complete college degrees and earn commissions in the Navy. The program increased enrollment to 1040 trainees, which was gradually reduced as V-12 was phased out after World War II. Since its founding, the unit has commissioned over 3,000 officers.

In 1980, the original Naval Armory was torn down to make way for the Edge Athletic Center, and the NROTC Unit moved to the then ceramic engineering building on the corner of Bobby Dodd Way and Fowler Street. In 2007, the unit moved again into the 2nd floor of the O’Keefe building, which also currently houses the Army and Air Force ROTC units. The new building contains many artifacts from the old facilities, including an eagle figurehead from the bow of the battleship USS Georgia (circa 1910),  her ship’s bell and various items from her bridge, the 4-in. gun earlier used for training, a ship’s wheel from the USS St. Louis, and timber from the USS Constitution - the Navy’s oldest currently commissioned vessel.

In 2018, Kennesaw State University’s Southern Polytechnic College of Engineering and Engineering Technology joined the Naval Reserve Officer Training Corps Unit – Georgia Institute of Technology, which includes midshipmen from Georgia State University along with it. It is a part of the Atlanta Consortium, along with the NROTCU Morehouse, comprised of Morehouse College, Spelman College, and Clark Atlanta University.