Previously, if no `target_cfg` flag was used, the `target_cfg` option
would not get removed from the opts dict, causing an error when it got
passed through to `WebSocketProxy`. Now we always remove it.
Fixes#168.
Token plugins provide a generic interface for transforming a token
into a `(host, port)` tuple.
The plugin name is specified using the '--token-plugin' option,
and may either be the name of a class from `websockify.token_plugins`,
or a fully qualified python path to the token plugin class (see below).
An optional plugin parameter can be specified using the '--token-source'
option (a value of `None` will be used if no '--token-source' option is
passed).
Token plugins should inherit from `websockify.token_plugins.BasePlugin`,
and should implement the `lookup(token)` method. The value of the
'--token-source' option is available as `self.source`.
Several plugins are included by default. The `ReadOnlyTokenFile`
and `TokenFile` plugins implement functionality from '--target-config'
(with the former only reading the file(s) once, and the latter reading
them every time). The 'BaseTokenAPI' plugin fetches the value from
an API, returning the result of `process_result(response_object)`.
By default, `process_result` simply returns the text of the response,
but may be overriden. The `JSONTokenAPI` does just this, returning
the 'host' and 'port' values from the response JSON object.
The old '--target-config' option is now deprecated, and maps to the
`TokenFile` plugin under the hood.
Also-Authored-By: James Portman (@james-portman)
Closes#157
* commit '477dce6cf86d61b20a394f3cbf3170e60d199658':
websocket: use python logging module
websocket: fix exception statement introduced by comment 903e3f06ee557
Adapted to new standard SocketServer RequestHandler design. For
example, this means that self.i_am_client is not needed.
* commit '4459824cc8196ad78fe9258b6c560ad46fe4cd52':
websocket: do not exit at the middle of process
websocket: restore signals after processing
websocket: support SIGTERM as exit signal
* commit '6d27b5d321978586ea1601f757ead73dfba03da7':
Add 2 arguments to websockify.WSRequestHandler
As of now, only implemented the first command; see #83 for details.
Unit test data will now go to a temporary dir that will be deleted
once the test completes. The unit tests also setup a logger which
will persist so that it can be inspected once tests complete.
Also fixes a bug where instance var is missing from decode_hybi()
Co-authored-by: natsume.takashi@lab.ntt.co.jp
To run the unit tests just run tox from the top
level directory which will try to run unit tests
for most versions of python. Requires tox to be
installed. To run tox for a specifice env, run
tox -e<env> e.g. for python 2.7 run 'tox -epy27'.
Co-authored-by: natsume.takashi@lab.ntt.co.jp
This commit should fix#101 by enabling a special SIGCHLD
handler for when multiprocessing is in use. The handler
simply calls `multiprocessing.active_children()` (which in
turn calls `_cleanup()`) upon receiving a SIGCHLD. Now,
the `fallback_SIGCHLD` is only called when `multiprocessing`
is not in use. See also #95.
TCP_KEEPALIVE is now enabled by default. Settings for
KEEPCNT, KEEPINTVL and KEEPIDLE can be supplied when
creating WebSocketServer and KEEPALIVE can also be
disabled if required.
Also adds new unit test for testing.
Co-authored-by: natsume.takashi@lab.ntt.co.jp
WebSocketServer is a library module, as such, it cannot assume it can
write output to process stdout.
Python logging module is designed in order to allow subscribers to
handle the output out of modules. It is simple and generic mechanism to
separate between data producer and data handling.
Python logging API also has the nature of log level, so the verbose
parameter can probably be obsoleted in favor of logging level. And of
course the logging API has built in support for exception tracebacks, no
need for manual format.
Per upstream request a wrapper is created around python logging to
enable shorter statements and optional replacement.
Add --traffic parameter for traffic specific debug, this is required as
it uses direct unformatted stdout output.
Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>
WebSocketServer is a library module, as such it should not exit process
but return from a method, allowing the caller to execute process show
down.
Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>
WebSocketServer is a library module, as such it should try to restore state
after processing, to allow caller to resume normal operation.
Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>
The WebSocketProxy class is usable for creating derived applications
with different logic, especially for the target validation.
Current code assumes that target is a path while in other implementation
it can be object that is loaded at initialization.
This change moves the conversion to absolute path into main function, so
that the WebSocketProxy class will not make that assumption.
Signed-off-by: Alon Bar-Lev <alon.barlev@gmail.com>
This patch adds 2 arguments to websockify.WSRequestHandler for security:
* file_only: returns 404 response if non-file contents are requested.
Required to disable directory listing.
* no_parent: returns 403 response if contents out of the web root are
requested. Required to disable directory traversal.
TypeError: exceptions must be old-style classes or derived from
BaseException, not str
Thus, we are not allowed to raise a string. Raise Exception instead.
refactoring. Basically, we are dividing WebSocketServer into two
classes: One request handler following the SocketServer Requesthandler
API, and one optional server engine. The standard Python SocketServer
engine can also be used.
websocketproxy.py has been updated to match the API change. I've also
added a new option --libserver in order to use the Python built in
server instead.
I've done a lot of testing with the new code. This includes: verbose,
daemon, run-once, timeout, idle-timeout, ssl, web, libserver. I've
tested both Python 2 and 3. I've also tested websocket.py in another
external service.
Code details follows:
* The new request handler class is called WebSocketRequestHandler,
inheriting SimpleHTTPRequestHandler.
* The service engine is called WebSocketServer, just like before.
* do_websocket_handshake: Using send_header() etc, instead of manually
sending HTTP response.
* A new method called handle_websocket() upgrades the connection to
WebSocket, if requested. Otherwise, it returns False. A typical
application use is:
def do_GET(self):
if not self.handle_websocket():
# handle normal requests
* new_client has been renamed to new_websocket_client, in order to
have a better name in the SocketServer/HTTPServer request handler
hierarchy.
* Note that in the request handler, configuration variables must be
provided by the "server" object, ie self.server.target_host.
Move around functions and methods, so that connection-related and
server-related stuff are close together.
This patch just moves things around - it does not change anything at
all. This can be verified with:
git diff websocket.py | grep ^- | cut -c 2- | sort > removed
git diff websocket.py | grep ^+ | cut -c 2- | sort > added
diff -u removed added
If WebSocketServer is used as a library with run_once or timeout, then
cleanup the socket listener socket so that when start_server returns
(due to run_once or timeout) then port is freed up.
Should at least mostly address this issue:
https://github.com/kanaka/websockify/issues/63
The problem is that the target in the test case often immediately
closed the target after sending the data. In this case the last data
received from the target is never sent to the client because the
client is not in the list of sockets being selected against (because
it is only added if their is pending data). By moving the client
conditionals first, we give the client socket a chance to be sent data
before the target is detected as close and we terminate the loop.
Fix recording so that it records the actual payload bytes sent to the
client. This means that if the client and server and using base64
encoding then the captured data will still be base64 encoded. However,
data received from the client is unmasked when recorded. Note that
this is not done efficiently; when recording, client data is unmasked
twice (once for sending on to the target and once for recording). This
could be made more efficient but that would require a refactor of how
frame reception and unmasking works and recording is not considered
a performance sensitive mode.
Old users of the websockify library used websockify.WebSocketProxy
The refactor into a module unneccessarily broke this import. The
current imports in __init__.py don't actually do anything so this
patch changes the import to import everything so that the old import
still works. This appears to be the original intention of the existing
import statements.
Python ignores SIGPIPE on startup, because it prefers to check every
write and raise an IOError exception rather than taking the signal. Most
Unix subprocesses don't expect to work this way. This patch (adapted
from Colin Watson's post at http://tinyurl.com/2a7mzh5) sets SIGPIPE
back to the default action.
Make websockify subdirectory and move websocket.py ->
websockify/websocket.py and websockify ->
websockify/websocketproxy.py. Create a ./run script that launches
websockify as before (unfortunately can't have a websockify script at
the same level since this is now a directory). Make websockify.py
a symlink to ./run. Once the package is installed, the main launch
script will be /usr/bin/websockify.
This makes it easier to package up websockify as a python module.
setup.py should now properly install websockify as a module.
Note that to include the base websocket module/class you will now do:
import websockify.websocket
#OR
from websockify.websocket import WebSocketServer
To import the full websocket proxy functionality:
import websockify.websocketproxy
#OR
from websockify.websocket import WebSocketProxy
This will also help with startup speed slightly because the code in
websocketproxy will now be byte compiled since it is no longer in the
main invocation script.
Config file syntax is now like this:
----------------------
# Comments and blank lines are allow
token1: host1:port1
token2: host2:port2
----------------------
websocket.py has no concept of target/proxy so any target processing
should happen in websockify itself.
Also:
- remove URL parsing imports from websocket.py since they are not
needed with SimpleHTTPRequestHandler doing the parsing.
- read the absolute path of the target_list file on startup since the
--web option will change directories if set.
The --target-list option is used to pass a configuration file on websockify start.
This file contains entries in the form host:port:token, one per line.
When a new connection is open (from noVNC for example), the url should look like:
ws://websockify_host:websockify_port/?token=ABCD
The token is then extracted and checked against the entries in the configuration file.
When the token matches, the connection is proxied to the host:port indicated in the file.
If the configuration file is a folder, then all the entries from files in it are read.
Add the option "-6, --prefer-ipv6". When set, the 'prefer_ipv6' flag in
websocket.py is used so that 'source_addr' is resolved to IPv6 if
available. If 'source_addr' is not set, binds to [::].
- The --web option changes directory so the wrap mode needs to get an
absolute path to the rebinder.
- Also, use long instead of int in rebind.c so avoid compile warnings.
- Rename unix socket option to '--unix-target' to be consistent with
'--ssl-target' which is an analogous switch.
- Fix normal socket target mode which was broken after merge.
- Normalize/fix output for SSL, unix socket and wrap command modes.
- --run-once will exit after handling a single WebSocket connection
(but not ater flash policy or normal web requests).
- --timeout TIME will stop listening for new connections after exit
after TIME seconds (the master process shuts down). Existing
WebSocket connections will continue but once all connections are
closed all processes will terminate.
Use rsplit(':', 1) instead of rpartition(':') in port argument
processing.
python2.4 may or may not work with HyBi which requires the numpy and
ctypes modules.
WebSocketServer.socket() is a static method takes a host and port and
an optional connect parameter. If connect is not set then it returns
a socket listening on host and port. If connect is set then
a connection will be made host and port and the socket returned. This
has IPv6 support like the addrinfo method it replaces.
Also, prefer IPv4 resolutions if they are in the list. This can be
overriden to prefer IPv6 resolutions for the same host using the
optional prefer_ipv6 parameter.
- fix addrinfo to accept empty host as localhost
- use correct host variable in addrinfo error message
- accept HyBi 7, 8 and 9. No difference for now.
- send close buffer correctly.
WebSocketServer(..., record='FILE_PREFIX')
The reocrd parameter will turn on recording of all messages sent
to and from the client. The record parameter is a file prefix. The
full file-name will be the prefix with an extension '.HANDLER_ID'
based on the handler ID.
Recording required some restructing of the encode and decode function
to return more information so that the recording functions can record
just the payload data and ignore the WebSockets framing/headers.
Caveats:
- Not all messages recorded as sent to the client were necessarily
received by the client. For example, if several messages are queued
for the client, but the connection is shutdown before the messages
are actually sent, these queued messages will still appear in the
recording.
- If the server is also handling HTTP requests then the handler ID
extensions for the recorded files will be monotonic but not
contiguous because only WebSocket connections are recorded, not HTTP
requests.
Multiprocessing:
- Switch to using multiprocessing module for python >= 2.6. For python
2.4 continue to use the os.fork() method.
- Move the new_client creation into top_new_client method to enable
multiprocessing refactor.
- Only do SIGCHLD handling for os.fork/python 2.4. When doing our own
SIGCHLD handling under python 3.0, we can run into a python futex
hang when reloading a web page rapidly. Multiprocessing does it's
own child reaping so we only need it with os.fork().
Python 3.0:
- Modify imports to reflect new locations: StringIO from io,
SimpleHTTPRequestHandler from http.server, urlsplit from
urllib.parse.
- Convert all print statements to print() calls. This also means no
comma parameter idiom and only using string formatting.
- Define b2s (bytes-to-string) and s2b (string-to-bytes) which are
no-ops on python versions prior to python 3. In python 3 these do
the conversion between string and bytes.
- Use compatible try/except method. Exception variable must be
extracted using sys.exc_info() rather than as part of the except
statement.
Python 2.4:
- Now degrades more gracefully if ssl module is not found. It will
still run, but will refuse SSL connections.
- Doesn't support HyBi-07 version due to numpy and struct.unpack_from
requirement.
- Add initial IETF-07 (HyBi-07) protocol version support. This version
still uses base64 encoding since the API for binary support is not
yet finalized.
- Move socket send and recieve functions into the WebSocketServer
class instead of having the sub-class do this. This simplifies
sub-classes somewhat. The send_frame routine now returns the number
of frames that were unable to be sent. If this value is non-zero
then the sub-class should call again when the socket is ready until
the pending frames count is 0.
- Do traffic reporting in the main class instead.
- When the client is HyBi style (i.e. IETF-07) then use the
sub-protocol header to select whether to do base64 encoding or
simply send the frame data raw (binary). Update include/websock.js
to send a 'base64' protocol selector. Once the API support binary,
then the client will need to detect this and set the protocol to
'binary'.
Wait 3 seconds for the client to send something. If no data is
available within 3 seconds then close the connection. It's probably
a non-WebSockets client that is waiting for the server to say
something first.
Aric Stewart
Solly Ross
Solly
Rafael Folco
Radek Podgorny
Matthew M. Boedicker
samhed
Peter Åstrand (astrand)
Edward Hope-Morley
Joel Martin
Steven Schmid
Alon Bar-Lev
Jason Dillaman
NTT Data OpenStackers
Vishvananda Ishaya
mostafa sameh
Aric Stewart
Hector Sanjuan
Alexandre Sicard
Karim Allah Ahmed
Daniel Shields
Cédric de Saint Martin
Vivien Alger