WebSocketServer(..., record='FILE_PREFIX')
The reocrd parameter will turn on recording of all messages sent
to and from the client. The record parameter is a file prefix. The
full file-name will be the prefix with an extension '.HANDLER_ID'
based on the handler ID.
Recording required some restructing of the encode and decode function
to return more information so that the recording functions can record
just the payload data and ignore the WebSockets framing/headers.
Caveats:
- Not all messages recorded as sent to the client were necessarily
received by the client. For example, if several messages are queued
for the client, but the connection is shutdown before the messages
are actually sent, these queued messages will still appear in the
recording.
- If the server is also handling HTTP requests then the handler ID
extensions for the recorded files will be monotonic but not
contiguous because only WebSocket connections are recorded, not HTTP
requests.
Ticket #2: https://github.com/kanaka/websockify/issues/2 - win32
support. The 'resource' module is not available under Windows. We only
use it for daemonizing so make it optional and disable daemonizing
support on Windows for now.
Also, refactor how the optional imports to turn them into data instead
of code and iterate through them.
Add early warnings to indicate when modules are missing and
functionality is disabled.
Multiprocessing:
- Switch to using multiprocessing module for python >= 2.6. For python
2.4 continue to use the os.fork() method.
- Move the new_client creation into top_new_client method to enable
multiprocessing refactor.
- Only do SIGCHLD handling for os.fork/python 2.4. When doing our own
SIGCHLD handling under python 3.0, we can run into a python futex
hang when reloading a web page rapidly. Multiprocessing does it's
own child reaping so we only need it with os.fork().
Python 3.0:
- Modify imports to reflect new locations: StringIO from io,
SimpleHTTPRequestHandler from http.server, urlsplit from
urllib.parse.
- Convert all print statements to print() calls. This also means no
comma parameter idiom and only using string formatting.
- Define b2s (bytes-to-string) and s2b (string-to-bytes) which are
no-ops on python versions prior to python 3. In python 3 these do
the conversion between string and bytes.
- Use compatible try/except method. Exception variable must be
extracted using sys.exc_info() rather than as part of the except
statement.
Python 2.4:
- Now degrades more gracefully if ssl module is not found. It will
still run, but will refuse SSL connections.
- Doesn't support HyBi-07 version due to numpy and struct.unpack_from
requirement.
Instead of doing crazy things with file-descriptors in
SplitHTTPHandler, WSRequestHandler detects a WebSockets upgrade and
returns 101 as the 'last_code'. In addition to avoiding funkiness with
file-descriptors, this allows use of the already parsed headers and
removal of the parse_handshake routine.
This also makes it easier to explore adding python 3.X support
(https://github.com/kanaka/websockify/issues/1) since the
file-descriptor/string splitting is very difficult to integrate with
python 3.X.
- Add initial IETF-07 (HyBi-07) protocol version support. This version
still uses base64 encoding since the API for binary support is not
yet finalized.
- Move socket send and recieve functions into the WebSocketServer
class instead of having the sub-class do this. This simplifies
sub-classes somewhat. The send_frame routine now returns the number
of frames that were unable to be sent. If this value is non-zero
then the sub-class should call again when the socket is ready until
the pending frames count is 0.
- Do traffic reporting in the main class instead.
- When the client is HyBi style (i.e. IETF-07) then use the
sub-protocol header to select whether to do base64 encoding or
simply send the frame data raw (binary). Update include/websock.js
to send a 'base64' protocol selector. Once the API support binary,
then the client will need to detect this and set the protocol to
'binary'.
Primary change is removal of FABridge interface.
Seems to improve overall latency by perhaps 10%. Also, the slowdown
over time in Opera is about half as bad (but still there).
Convert latency test to use include/websock.js instead of direct
WebSockets handling.
Add support for configuring the maximum bufferedAmount. This allows us
to configure it high enough to get around a bug in bufferedAmount
reporting in Opera.
Add some latency test results for Opera 11 with WebSockets turned on.
- Only delay sending data if bufferedAmount is greater than 1000.
This seems to match the intention of the spec better. bufferedAmount
does not mean that we can't send, it's just an indication that the
network is becoming saturated. But Opera 11 native WebSockets seems to
have a bug that bufferedAmount isn't set back to zero correctly so
- websock.send returns true/false.
If all send data was flushed from the send queue then return true,
otherwise false. This doesn't mean the data won't be sent, just that
it wasn't sent this time and is queued.
20f837425d4 changes to a single event stream handler fixing the
recursive call errors in firefox and Opera.
Also, pull web-socket-js fix from noVNC
Related to this issue:
https://github.com/gimite/web-socket-js/issues/#issue/50
This prevents the "Uncaught exception: TypeError:
'this.__handleEvents' is not a function" everytime the timer fires.
Wait 3 seconds for the client to send something. If no data is
available within 3 seconds then close the connection. It's probably
a non-WebSockets client that is waiting for the server to say
something first.
The Websock object from websock.js is similar to the standard
WebSocket object but Websock enables communication with raw TCP
sockets (i.e. the binary stream) via websockify. This is accomplished
by base64 encoding the data stream between Websock and websockify.
Websock has built-in receive queue buffering; the message event
does not contain actual data but is simply a notification that
there is new data available. Several rQ* methods are available to
read binary data off of the receive queue.
wswrapper:
Getting the wswrapper.c LD_PRELOAD model working has turned out to
involve too many dark corners of the glibc/POSIX file descriptor
space. I realized that 95% of what I want can be accomplished by
adding a "wrap command" mode to wsproxy.
The code is still there for now, but consider it experimental at
best. Minor fix to dup2 and add dup and dup3 logging.
wsproxy Wrap Command:
In wsproxy wrap command mode, a command line is specified instead
of a target address and port. wsproxy then uses a much simpler
LD_PRELOAD library, rebind.so, to move intercept any bind() system
calls made by the program. If the bind() call is for the wsproxy
listen port number then the real bind() system call is issued for
an alternate (free high) port on loopback/localhost. wsproxy then
forwards from the listen address/port to the moved port.
The --wrap-mode argument takes three options that determine the
behavior of wsproxy when the wrapped command returns an exit code
(exit or daemonizing): ignore, exit, respawn.
For example, this runs vncserver on turns port 5901 into
a WebSockets port (rebind.so must be built first):
./utils/wsproxy.py --wrap-mode=ignore 5901 -- vncserver :1
The vncserver command backgrounds itself so the wrap mode is set
to "ignore" so that wsproxy keeps running even after it receives
an exit code from vncserver.
wstelnet:
To demonstrate the wrap command mode, I added WebSockets telnet
client.
For example, this runs telnetd (krb5-telnetd) on turns port 2023
into a WebSockets port (using "respawn" mode since telnetd exits
after each connection closes):
sudo ./utils/wsproxy.py --wrap-mode=respawn 2023 -- telnetd -debug 2023
Then the utils/wstelnet.html page can be used to connect to the
telnetd server on port 2023. The telnet client includes VT100.js
(from http://code.google.com/p/sshconsole) which handles the
terminal emulation and rendering.
rebind:
The rebind LD_PRELOAD library is used by wsproxy in wrap command
mode to intercept bind() system calls and move the port to
a different port on loopback/localhost. The rebind.so library can
be built by running make in the utils directory.
The rebind library can be used separately from wsproxy by setting
the REBIND_OLD_PORT and REBIND_NEW_PORT environment variables
prior to executing a command. For example:
export export REBIND_PORT_OLD="23"
export export REBIND_PORT_NEW="65023"
LD_PRELOAD=./rebind.so telnetd -debug 23
Alternately, the rebind script does the same thing:
rebind 23 65023 telnetd -debug 23
Other changes/notes:
- wsproxy no longer daemonizes by default. Remove -f/--foreground
option and add -D/--deamon option.
- When wsproxy is used to wrap a command in "respawn" mode, the
command will not be respawn more often than 3 times within 10
seconds.
- Move getKeysym routine out of Canvas object so that it can be called
directly.
- add dup2 functionality. This requires adding a ref cnt to the
_WS_connections structure so that we only free the structure once
all dup'd referenced are closed. Also, refactor malloc and free of
connection structure into _WS_alloc and _WS_free.
- allow select to accept a NULL timeout value which means sleep
forever instead of segfaulting.
- fix some compile warnings related to ppoll definition.
- move some WebSockets related html test pages into utils and symlink
them from tests.
Moved websocket.py code into a class WebSocketServer. WebSockets
server implementations will sub-class and define a handler() method
which is passed the client socket after. Global variable settings have been
changed to be parameters for WebSocketServer when created.
Subclass implementations still have to handle queueing and sending but
the parent class handles everything else (daemonizing, websocket
handshake, encode/decode, etc). It would be better if the parent class
could handle queueing and sending. This adds some buffering and
polling complexity to the parent class but it would be better to do so
at some point. However, the result is still much cleaner as can be
seen in wsecho.py.
Refactored wsproxy.py and wstest.py (formerly ws.py) to use the new
class. Added wsecho.py as a simple echo server.
- rename tests/ws.py to utils/wstest.py and add a symlink from
tests/wstest.py
- rename tests/ws.html to tests/wstest.html to match utils/wstest.py.
- add utils/wsecho.py
- add tests/wsecho.html which communicates with wsecho.py and simply
sends periodic messages and shows what is received.
Algervivien
Joel Martin
Cédric de Saint Martin
Chris Custine