To solve the low throughput problem in radio link control (RLC) layer caused by large delay in GEO mobile satellite communication when using existing retransmission mechanism (SR-ARQ), a HARQ schema applied to GEO mobile satellite communication (G-HARQ) in RLC layer is proposed. Then, the G-HARQ and SR-ARQ mechanism of RLC layer is simulated in GEO mobile commutation platform based on QualNet. Simulation result shows that the G-HARQ retransmission mechanism can reduce the number of ACK/NACK control message generated when recover error RLC-Block in RLC acknowledge mode, which can ameliorate the pear-to-pear delay in RLC layer of GEO mobile communication protocol, and improve the system throughput.