Turns out when Windows is running in QEMU and a window scroll happens,
there are lots of little hextile rects sent. This is slow in noVNC.
- Some recording/playback improvement.
- Add test harness to drive playback of recordings.
- By pulling off the rect header in one chunk we get a 3X speedup in
Chrome and a 20% speedup in firefox (specifically for the scroll
test).
- Also, get rid of some noise from creating timers for handle_message.
Check to make sure there isn't already a pending timer first.
This is very usefull when you need to open a new window (with a new document) from javascript,
without having to reload the script.js.
(cherry picked from commit 8ded53c1de06d01e50d58543c19e73926f0fbbd4)
Signed-off-by: Joel Martin <github@martintribe.org>
New API:
To use the RFB object, you now must instantiate it (this allows more
than one instance of it on the same page).
rfb = new RFB(settings);
The 'settings' variable is a namespace that contains initial default
settings. These can also be set and read using 'rfb.set_FOO()' and
'rfb.get_FOO()' where FOO is the setting name. The current settings
are (and defaults) are:
- target: the DOM Canvas element to use ('VNC_canvas').
- encrypt: whether to encrypt the connection (false)
- true_color: true_color or palette (true)
- b64encode: base64 encode the WebSockets data (true)
- local_cursor: use local cursor rendering (true if supported)
- connectTimeout: milliseconds to wait for connect (2000)
- updateState: callback when RFB state changes (none)
- clipboardReceive: callback when clipboard data received (none)
The parameters to the updateState callback have also changed. The
function spec is now updateState(rfb, state, oldstate, msg):
- rfb: the RFB object that this state change is for.
- state: the new state
- oldstate: the previous state
- msg: a message associate with the state (not always set).
The clipboardReceive spec is clipboardReceive(rfb, text):
- rfb: the RFB object that this text is from.
- text: the clipboard text received.
Changes:
- The RFB and Canvas namespaces are now more proper objects. Private
implementation is no longer exposed and the public API has been made
explicit. Also, instantiation allows more than one VNC connection
on the same page (to complete this, DefaultControls will also need
this same refactoring).
- Added 'none' logging level.
- Removed automatic stylesheet selection workaround in util.js and
move it to defaultcontrols so that it doesn't interfere with
intergration.
- Also, some major JSLinting.
- Fix input, canvas, and cursor tests to work with new model.
If cursor Data URI scheme detection threw an exception, it would cause
canvas initialization to fail. cursor detection exceptions should just
disable local cursor change support, not cause canvas init to fail.
Uses the CSS "scale()" operation. The main problem is that the DOM
container is not rescaled, only the size of the displayed content
within it so there will need to be some sort of mechanism to handle
this better so other elements reflow to the new size. Or it might just
not work and be removed later. The zoom property seems to do the right
behavior, but it's not widely supported. Worth exploring though.
After each complete framebufferUpdate, set a short timer to continue
processing the receive queue. This gives other events a chance to
fire. Especially important when noVNC is integrated into another
website.
noVNC was never processing more than one framebufferUpdate message per
onmessage event. If noVNC receives an incomplete framebufferUpdate and
then receives the rest of the framebufferUpdate plus another complete
framebufferUpdate, then it will fall permanently behind.
If there is more to process after a completed framebufferUpdate, then
execute normal_msg again.
All the render routines must return false if there is not enough data
in the receive queue to process their current update, and true
otherwise.
Move the whole RFB object to rfb.js. vnc.js is now just the loader
file. This allows an integrating project to easily replace vnc.js with
an alternate loader mechanism (or just do it directly in the html
file). Thanks for the idea primalmotion (http://github.com/primalmotion).
Also, JSLint the various files.
The following API changes may affect integrators:
- Settings have been moved out of the RFB.connect() call. Each
setting now has it's own setter function: setEncrypt, setBase64,
setTrueColor, setCursor.
- Encrypt and cursor settings now default to on.
- CSS changes:
- VNC_status_bar for input buttons switched to a element class.
- VNC_buttons split into VNC_buttons_right and
VNC_buttons_left
- New id styles for VNC_settings_menu and VNC_setting
Note: the encrypt, true_color and cursor, logging setting can all be
set on load using query string variables (in addition to host, port
and password).
Client cursor (cursor pseudo-encoding) support has been polished and
activated.
The RFB settings are now presented as radio button list items in
a drop-down "Settings" menu when using the default controls.
Also, in the settings menu is the ability to select between alternate
style-sheets.
Cookie and stylesheet selection support added to util.js.
To change the appearance of the cursor, we use the CSS cursor style
and set the url to a data URI scheme. The image data sent via the
cursor pseudo-encoding has to be encoded to a CUR format file before
being used in the data URI.
During Canvas initialization we try and set a simple cursor to see if
the browser has support. Opera is missing support for data URI scheme
in cursor URLs.
Disabled for now until we have a better way of specifying settings
overall (too many settings for control bar now).
Add new states 'loaded', 'connect' and 'fatal':
- Loaded state is first page state. Pass WebSockets mode message using
this state.
- Connect indicates that the user has issued a "connect" but we
haven't gotten an WebSockets onopen yet.
- Fatal is a condition that indicates inability to continue on: right
now, lack of WebSockets/Flash or non-working canvas.
Move much of the actual state transition code into updateState.
Handle 'password' state better in default_controls.js; instead of
disconnecting, prompt for password to send.
Add comments to updateState indicating possible states.
Util.Debug, Util.Info, Util.Warn, Util.Error routines instead of
direct calls to console.*. Add "logging=XXX" query variable that sets
the logging level (default is "warn").
Logging values:
debug: code debug logging (many calls in performance path are also
commented for performance reasons).
info: informative messages including timing information.
warn: significant events
error: something has gone wrong
The problem is, you can't set WebSocket.__swfLocation before you load
web_socket.js (because it creates the WebSocket global), but you also
can't reliably set WebSocket.__swfLocation after because if you are
doing dynamic script file includes then the onload (i.e.
WebSocket.__initialize) may fire before you have a chance to set
Websocket.__swfLocation.
Add message/state pollling in web-socket-js. Since Opera tends to drop
message events, we can dramatically increase performance by polling
every now for message event data.
Also, add more direct calls to update readyState so that it's not
missed when Opera drops events.
When using web-socket-js, the onopen event may happen inline so the
caller may not have time to set onopen before the event fires. In this
case set a short timeout and try again. In particular this affects
Opera most of the time.
Also, to get around Opera event droppings, always read the readyState
directly instead of relying on the local readyState variable to be
correct (which it isn't if stateChange event were dropped).
All browsers with Canvas imageData are faster with JS ops instead of
canvas ops. This gives significant performance improvement in Opera.
Except for missing web-socet-js message notifications, Opera 10.60 is
now faster than firefox 3.5.
Instead of relying on FABridge AS -> JS event delivery, we just use
the events to notify JS of pending data. The message handler then
calls the AS readSocketData routine which sends back an array of
the pending WebSocket frames.
There is still a minor bug somewhere that happens after the first
connect where the web-socket-js throws an "INVALID_STATE_ERR: Web
Socket connection has not been established". But, Opera is now usable
and we should be able to drop the packet sequence numbering and
re-ordering code.
Another minor issue to better support Opera is to move JS script
includes to the <head> of the page instead of after the body.
Interesting. Enough has changed in the Canvas tile operations, that
Canvas.prefer_js=true is better for firefox/gecko too. Approximately
2X improvement in firefox for large hextile renders.
Looks like disabling web-socket-js debug messages by default that we
get a minor speedup.
Python proxy should support both 75 and 76 (00) modes. Also, update ws
test to more reliably hit the WebSockets ordering/drop issue.
Some default_controls.js jslinting.
Needs to be some modularity between controls you probably always want
(like sending CtrlAltDel) and how the interface is presented and
controlled.
Some basic functions from mootools implemented in util.js.
Also, some more DOM separation. Move clipboard focus logic into
default_controls and canvas and out of vnc.js.
JSLint cleanup.
Move DOM manipulation into include/default_controls.js and update
vnc.html to use it.
Add an example vnc_auto.html which automatically connects using
parameters from the query string and doesn't use default_controls.js.
Reorder functions in vnc.js to put external interface functions at the
top of the RFB namespace.
In colourMap mode there are 256 colours in a colour palette sent from
the server via the SetColourMapEntries message. This reduces the
bandwidth by about 1/4. However, appearance can be somewhat less than
ideal (pinks instead of gray, etc).
It also increases client side rendering performance especially on
firefox. Rendering a full 800x600 update takes about 950ms in
firefox on my system compared to about 1400ms. Round-trip time for
a full frame buffer update is even better on firefox (due to
performance of the flash WebSocket emulator). Reduced from about
1800ms to 1100ms on firefox (for 800x600 full update).
Also add a wsencoding test client/server program to test send a set of
values between client and server and vice-versa to test encodings.
Not turned on by default.
Add support for encode/decode of UTF-8 in the proxy. This leverages
the browser for decoding the WebSocket stream directly instead of
doing base64 decode in the browser itself.
Unfortunately, in Chrome this has negligible impact (round-trip time
is increased slightly likely due to extra python processing).
In firefox, due to the use of the flash WebSocket emulator the
performance is even worse. This is because it's really annoying to get
the flash WebSocket emulator to properly decode a UTF-8 bytestream.
The problem is that the readUTFBytes and readMultiByte methods of an
ActionScript ByteArray don't treat 0x00 correctly. They return
a string that ends at the first 0x00, but the index into the ByteArray
has been advanced by however much you requested.
This is very silly for two reasons: ActionScript (and Javascript)
strings can contain 0x00 (they are not null terminated) and second,
UTF-8 can legitimately contain 0x00 values. Since UTF-8 is not
constant width there isn't a great way to determine if those methods
in fact did encounter a 0x00 or they just read the number of bytes
requested.
Doing manual decoding using readUTFByte one character at a time slows
things down quite a bit. And to top it all off, those methods don't
support the alternate UTF-8 encoding for 0x00 ("\xc0\x80"). They also
just treat that encoding as the end of string too.
So to get around this, for now I'm encoding zero as 256 ("\xc4\x80")
and then doing mod 256 in Javascript. Still doesn't result in much
benefit in firefox.
But, it's an interesting approach that could use some more exploration
so I'm leaving in the code in both places.
The purpose of the code is to be incorporated into other web projects
(whether those are free or not). AGPL prevents combination with other
HTML and javascript that is under a weaker (or proprietary) license.
Better would be a lesser AGPL, but there is not GNU standard for that.
So LGPL-3 meets most of my requirements. If somebody modifies the
actual client code and conveys it, then they must release the changes
under LGPL-3 also.
Add some implementation notes in docs/notes.
- By dereferencing the 'data' field of the imageData object before the
loop, the hextile performance on Chrome is down to 140ms or so for
a full 800x600 update. Still have to fall back to Canvas operations
for firefox.
- Fix RQ empty after reorder bug.
- Instead of onload override, move to RFB.load function that takes
a parameter for the target DOM ID. This allows the user to have
their own onload function.
- Add "VNC_" prefix to all element ID names. Only create DOM elements
if they don't already exist on the page, otherwise use the existing
elements.
- Move all styling to separate stylesheet.
- Use list model for control styling.
On the client side, this adds the as3crypto library to web-socket-js
so that the WebSocket 'wss://' scheme is supported which is WebSocket
over SSL/TLS.
Couple of downsides to the fall-back method:
- This balloons the size of the web-socket-js object from about 12K to 172K.
- Getting it working required disabling RFC2718 web proxy support
in web-socket-js.
- It makes the web-socket-js fallback even slower with the
encryption overhead.
The server side (wsproxy.py) uses python SSL support. The proxy
automatically detects the type of incoming connection whether flash
policy request, SSL/TLS handshake ('wss://') or plain socket
('ws://').
Also added a check-box to the web page to enable/disabled 'wss://'
encryption.
- All state/status updates go through updateState routine which
updates the status line also.
- Old firefox (and opera) don't support canvas createImageData, so use
getImageData as replacement.
- Add console.warn and console.error stubs so that firefox without
firebug doesn't crap out.
- If no WebSockets then error if no flash or if URL is location (flash
will refuse to load the object for security reasons).
- web-socket-js is from http://github.com/gimite/web-socket-js. It is
a flash object that emultates WebSockets.
Unfortunately, events (or packets) from the web-socket-js object can
get re-ordered so we need to know the packet order.
- So wsproxy.py prepends the sequence number of the packet when
sending.
- If the client receives packets out of order it queues them up and
scans the queue for the sequence number it's looking for until
things are back on track. Gross, but hey: It works!
- Also, add packet sequence checking to wstest.*
- DES encryption for VNC bit mirrors every bytes of the password. This
commit has a hard-coded mirrored password. Need to ask user and bit
mirror it.
- With image data across the wire it's Blue,Green,Red, so twiddle
things around a bit.
Joel Martin
primalmotion
Joel Martin