python是一门很强大且易用的脚本语言.一直很想好好学习python很久了,在阅读完了<python学习手册>后,算是在python方面入了门.由于墙的存在,想要很方便的访问国外网站就需要梯子,而shadowsocks是其中的比较简单而且好用的工具.用了一阵子之后,本着
学习一门语言的最好办法就是阅读好的开源项目源码
原则,就打算开坑学习shadowsocks的实现.
在阅读了一部分之后,看到了python中socket server的使用方法,十分简洁,遂做了以下记录.
Socket Server使用
SocketServer是python提供的实现socket server的模块.利用其提供的api,可以很方便快速的开发一个socket服务器.整体上来说,模块提供了四个使用的类:
- TCPServer
- UDPServer
- UnixStreamServer
- UnixDatagramServer
上述的四个类都是同步的,即意味着当有请求过来后,其将同步处理数据,处理完之后才能处理下一个请求.
编写一个Socket Server
编写一个SocketServer需要实现以下步骤
- 编写一个handler类,继承BaseRequestHandler,重写handle()方法
- 针对是TCP还是UDP,生成一个server对象
- 调用server对象的handle_request或者serve_forever方法
例子摘自python文档: “`python import SocketServer
class MyTCPHandler(SocketServer.BaseRequestHandler):
def handle(self):
# self.request is the TCP socket connected to the client
self.data = self.request.recv(1024).strip()
print "{} wrote:".format(self.client_address[0])
print self.data
# just send back the same data, but upper-cased
self.request.sendall(self.data.upper())
if __name__ == "__main__":
HOST, PORT = "localhost", 9999
# Create the server, binding to localhost on port 9999
server = SocketServer.TCPServer((HOST, PORT), MyTCPHandler)
# Activate the server; this will keep running until you
# interrupt the program with Ctrl-C
server.serve_forever()
用户端代码
import socket
import sys
HOST, PORT = "localhost", 9999
data = " ".join(sys.argv[1:])
# Create a socket (SOCK_STREAM means a TCP socket)
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
try:
# Connect to server and send data
sock.connect((HOST, PORT))
sock.sendall(data + "n")
# Receive data from the server and shut down
received = sock.recv(1024)
finally:
sock.close()
print "Sent: {}".format(data)
print "Received: {}".format(received)
handler中的self.request属性
对于不同类型的Server,request对象的类型并不相同
TCP: request为socket.socket对象,具体请查看文档
UDP: request为内容是(data, socket)的元组.
同步与异步
对于TCPServer和UDPServer来说,其在处理请求时,都是同步请求的.这意味着,只有一个请求处理完毕后,才能继续处理接下来的请求.如果请求处理需要很长时间,或者请求与Server之间有较多的交互,那同步处理就不大合适了.
在SocketServer模块中,提供了两种服务模型.
- ThreadingMixIn
- ForkingMixIn
顾名思义,ThreadingMixIn代表其在有新的请求时,创建一个新的线程,在该线程中处理请求.相对应的,ForkingMixIn表示在有新的请求时,创建一个新的进程,在该进程中处理请求.
对于该选择何种模型,可以考虑是否需要请求间数据共享.
#ThreadingMinIn例子
import socket
import SocketServer
import threading
class ThreadTCPRequestHandler(SocketServer.BaseRequestHandler):
def handle(self):
data = self.request.recv(1024)
cur_thread = threading.current_thread()
response = "{}: {}".format(cur_thread.name, data)
self.request.sendall(response)
class ThreadTCPServer(SocketServer.ThreadingMixIn, SocketServer.TCPServer):
pass
def client(ip, port, message):
sock = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
sock.connect((ip, port))
try:
sock.sendall(message)
response = sock.recv(1024)
print 'Received: {}'.format(response)
finally:
sock.close()
if __name__ == '__main__':
HOST, PORT = 'localhost', 0
server = ThreadTCPServer((HOST, PORT), ThreadTCPRequestHandler)
ip, port = server.server_address
server_thread = threading.Thread(target=server.serve_forever)
server_thread.daemon = True
server_thread.start()
print 'Server loop running in thread:', server_thread.name
#send request in thread
for i in range(3):
thread = threading.Thread(target=client, args=(ip, port, 'hello world from {}'.format(i)))
thread.start()
print 'thread {} start'.format(i)
实现探究
所谓
知其然,更要知其所以然
所以,我们也有必要可以一窥SocketServer的具体实现,同时,对于其性能也有一个较好的了解.SocketServer的实现代码在Lib/SocketServer.py.在*nix中可以通过locate命令,找到相应的路径.
在实际的实现中,类的继承关系如下图(摘自官方文档)
+------------+
| BaseServer |
+------------+
|
v
+-----------+ +------------------+
| TCPServer |------->| UnixStreamServer |
+-----------+ +------------------+
|
v
+-----------+ +--------------------+
| UDPServer |------->| UnixDatagramServer |
+-----------+ +--------------------+
BaseServer提供了几个值得关注的方法
- serve_forever
- _handle_request_noblock
- process_request
- shutdown_request
- shutdown
BaseServer.serve_forever:
def serve_forever(self, poll_interval=0.5):
#清除是否关闭的flag
self.__is_shut_down.clear()
try:
while not self.__shutdown_request:
#把自己加入到selelct的read池中,利用select轮询,轮询的超时时间是poll_interval
r, w, e = select.select([self], [], [], poll_interval)
if self in r:
self._handle_request_noblock()
finally:
self.__shutdown_request = False
self.__is_shut_down.set()
从代码中可以看出,SocketServer是采用了select来获取可以读取的socket进行处理.
BaseServer.handle_request_noblock:
def _handle_request_noblock(self):
#由于select返回的是可以读取的socket,所以理论上get_request能立即返回
try:
request, client_address = self.get_request()
except socket.error:
return
if self.verify_request(request, client_address):
try:
#process_request中调用finish_request,
#finish_request中生成一个Handler类,进行相应的处理
self.process_request(request, client_address)
except:
self.handle_error(request, client_address)
self.shutdown_request(request)
BaseServer.shutdown:
def shutdown(self):
#只有server_forever是在其他线程中调用的时候,才能调用shutdown,否则因为调用__is_shutdown.wait()去等待其他线程的事件而导致死锁
self.__shutdown_request = True
self.__is_shut_down.wait()
ThreadingMinIn.process_request: ThreadingMinIn重写了BaseServer中的process_request方法.
def process_request(self, request, client_address:
"""Start a new thread to process the request"""
t = threading.Thread(target = self.process_request_thread,
args = (request, client_addres))
if self.daemon_threds:
t.setDaemon(1)
t.start()
在处理请求的时候,调用threading模块的方法创建了一个新的线程,来处理请求.线程执行的是process_request_thread函数
def process_request_thread(self, request, client_address):
#和BaseServer一样,没什么修改
try:
self.finish_request(request, client_address)
#在处理完请求后就关闭该连接
self.shutdown_request(request)
except:
self.handle_error(request, client_address)
self.shutdown_request(request)
总结
从SocketServer这个模块中,我们可以看到python封装了很多已有的模块来提供快速开发,编写代码变得简单了很多.
最后发个听说的面试题吧
请问全国所有java程序员全部变成python程序员,能省多少行代码?
转载请注明：爱开源 » Python初探 Socket Server