New hooks for command processing in comm layer

Added phase specific hooks for queuing, queued, sending and sent phases of a command ("octoprint.comm.protocol.gcode.<phase>"). Removed old queuing phase hook and declared as obsolete hook in plugin manager to prevent plugins that depend on it from being enabled.

Adding those new hooks also necessitated refactoring the whole command processing, made it more modular and added phase specific handler functions that allow handling all blocking commands centrally for example.
This commit is contained in:
Gina Häußge 2015-05-28 17:06:24 +02:00
parent 1458503561
commit 61af59cca1
3 changed files with 206 additions and 111 deletions

View file

@ -40,37 +40,84 @@ octoprint.comm.protocol.action
:param str action: The parsed out action command, so for a ``line`` like ``// action:some_command`` this will be
``some_command``
.. _sec-plugins-hook-comm-protocol-gcode:
.. _sec-plugins-hook-comm-protocol-gcode-phase:
octoprint.comm.protocol.gcode
-----------------------------
octoprint.comm.protocol.gcode.<phase>
-------------------------------------
.. py:function:: hook(comm_instance, cmd, cmd_type=None, *args, **kwargs)
This describes actually four hooks:
Preprocess and optionally suppress a GCODE command before it is being sent to the printer.
* ``octoprint.comm.protocol.gcode.queuing``
* ``octoprint.comm.protocol.gcode.queued``
* ``octoprint.comm.protocol.gcode.sending``
* ``octoprint.comm.protocol.gcode.sent``
.. py:function:: hook(comm_instance, phase, cmd, cmd_type, gcode, *args, **kwargs)
Pre- and postprocess commands as they progress through the various phases of being sent to the printer. The phases
are the following:
* ``queuing``: This phase is triggered just before the command is added to the send queue of the communication layer. This
corresponds to the moment a command is being read from a file that is currently being printed. Handlers
may suppress or change commands or their command type here.
* ``queued``: This phase is triggered just after the command was added to the send queue of the communication layer.
No manipulation is possible here anymore (returned values will be ignored).
* ``sending``: This phase is triggered just before the command is actually being sent to the printer. Right afterwards
a line number will be assigned and the command will be sent. Handlers may suppress or change commands here. The
command type is not taken into account anymore.
* ``sent``: This phase is triggered just after the command was handed over to the serial connection to the printer.
No manipulation is possible here anymore (returned values will be ignored). A command that reaches the sent phase
must not necessarily have reached the printer yet and it might also still run into communication problems and a
resend might be triggered for it.
Hook handlers may use this to rewrite or completely suppress certain commands before they enter the send queue of
the communication layer. The hook handler will be called with the ``cmd`` to be sent to the printer as well as
the ``cmd_type`` parameter.
If the handler does not wish to handle the command, it should simply perform a ``return cmd`` as early as possible,
that will ensure that no changes are applied to the command.
If the handler wishes to suppress sending of the command altogether, it should return None instead. That will tell
OctoPrint that the ``cmd`` has been scraped altogether and not send anything.
More granular manipulation of the sending logic is possible by not just returning ``cmd`` (be it the original, a
rewritten variant or a None value) but also a 2-tuple ``(cmd, cmd_type)``. This
allows to also rewrite the ``cmd_type`` parameter used for sending.
the communication layer or before they are actually sent over the serial port, or to react to the queuing or sending
of commands after the fact. The hook handler will be called with the processing ``phase``, the ``cmd`` to be sent to
the printer as well as the ``cmd_type`` parameter used for enqueuing (OctoPrint will make sure that the send queue
will never contain more than one line with the same ``cmd_type``) and the detected gcode command (if it is one).
Defining a ``cmd_type`` other than None will make sure OctoPrint takes care of only having one command of that type
in its sending queue. Predefined types are ``temperature_poll`` for temperature polling via ``M105`` and
``sd_status_poll`` for polling the SD printing status via ``M27``.
``phase`` will always match the ``<phase>`` part of the implemented hook (e.g. ``octoprint.comm.protocol.gcode.queued``
handlers will always be called with ``phase`` set to ``queued``). This parameter is provided so that plugins may
utilize the same hook for mulitple phases if required.
Handlers are expected to return one of the following result variants:
* ``None``: Don't change anything. Note that Python functions will also automatically return ``None`` if
an empty ``return`` statement is used or just nothing is returned explicitely from the handler. Hence, the following
examples are all falling into this category:
.. code-block:: python
def one(*args, **kwargs):
print("I return None explicitly")
return None
def two(*args, **kwargs):
print("I just return without any values")
return
def three(*args, **kwargs):
print("I don't explicitly return anything at all")
Handlers which do not wish to modify (or suppress) ``cmd`` or ``cmd_type`` at all should use this option.
* A string with the rewritten version of the ``cmd``, e.g. ``return "M110"``. To avoid situations which will be
difficult to debug should the returned command be later changed to ``None`` (with the intent to suppress the
command instead but actually causing ``cmd`` and ``cmd_type`` to just staying as-is), this variant should be
entirely avoided by handlers.
* A 1-tuple consisting of a rewritten version of the ``cmd``, e.g. ``return "M110",``, or ``None`` in order to
suppress the command, e.g. ``return None,``. Handlers which wish to rewrite the command or to suppress it completely
should use this option.
* A 2-tuple consisting of a rewritten version of the ``cmd`` and the ``cmd_type``, e.g. ``return "M105", "temperature_poll"``.
Handlers which wish to rewrite both the command and the command type should use this option.
**Example**
The following hook handler replaces all ``M107`` ("Fan Off", deprecated) with an ``M106 S0`` ("Fan On" with speed
parameter)
parameter) upon queuing and logs all sent ``M106``.
.. onlineinclude:: https://raw.githubusercontent.com/OctoPrint/Plugin-Examples/master/rewrite_m107.py
:linenos:
@ -78,12 +125,14 @@ octoprint.comm.protocol.gcode
:caption: `rewrite_m107.py <https://github.com/OctoPrint/Plugin-Examples/blob/master/rewrite_m107.py>`_
:param object comm_instance: The :class:`~octoprint.util.comm.MachineCom` instance which triggered the hook.
:param str phase: The current phase in the command progression, either ``queuing``, ``queued``, ``sending`` or
``sent``. Will always match the ``<phase>`` of the hook.
:param str cmd: The GCODE command for which the hook was triggered. This is the full command as taken either
from the currently streamed GCODE file or via other means (e.g. user input our status polling).
:param str cmd_type: Type of command, ``temperature_poll`` for temperature polling or ``sd_status_poll`` for SD
:param str cmd_type: Type of command, e.g. ``temperature_poll`` for temperature polling or ``sd_status_poll`` for SD
printing status polling.
:return: A rewritten ``cmd``, a tuple of ``cmd`` and ``cmd_type``
or None to suppress sending of the ``cmd`` to the printer. See above for details.
:param str gcode: Parsed GCODE command, e.g. ``G0`` or ``M110``, may also be None if no known command could be parsed
:return: None, 1-tuple, 2-tuple or string, see the description above for details.
.. _sec-plugins-hook-comm-protocol-scripts:

View file

@ -32,7 +32,7 @@ from octoprint.util import deprecated
_instance = None
def plugin_manager(init=False, plugin_folders=None, plugin_types=None, plugin_entry_points=None, plugin_disabled_list=None,
plugin_restart_needing_hooks=None):
plugin_restart_needing_hooks=None, plugin_obsolete_hooks=None):
"""
Factory method for initially constructing and consecutively retrieving the :class:`~octoprint.plugin.core.PluginManager`
singleton.
@ -55,6 +55,8 @@ def plugin_manager(init=False, plugin_folders=None, plugin_types=None, plugin_en
plugin_restart_needing_hooks (list): A list of hook namespaces which cause a plugin to need a restart in order
be enabled/disabled. Does not have to contain full hook identifiers, will be matched with startswith similar
to logging handlers
plugin_obsolete_hooks (list): A list of hooks that have been declared obsolete. Plugins implementing them will
not be enabled since they might depend on functionality that is no longer available.
Returns:
PluginManager: A fully initialized :class:`~octoprint.plugin.core.PluginManager` instance to be used for plugin
@ -97,8 +99,18 @@ def plugin_manager(init=False, plugin_folders=None, plugin_types=None, plugin_en
plugin_restart_needing_hooks = [
"octoprint.server.http"
]
if plugin_obsolete_hooks is None:
plugin_obsolete_hooks = [
"octoprint.comm.protocol.gcode"
]
_instance = PluginManager(plugin_folders, plugin_types, plugin_entry_points, logging_prefix="octoprint.plugins.", plugin_disabled_list=plugin_disabled_list, plugin_restart_needing_hooks=plugin_restart_needing_hooks)
_instance = PluginManager(plugin_folders,
plugin_types,
plugin_entry_points,
logging_prefix="octoprint.plugins.",
plugin_disabled_list=plugin_disabled_list,
plugin_restart_needing_hooks=plugin_restart_needing_hooks,
plugin_obsolete_hooks=plugin_obsolete_hooks)
else:
raise ValueError("Plugin Manager not initialized yet")
return _instance

View file

@ -172,7 +172,14 @@ class MachineCom(object):
# hooks
self._pluginManager = octoprint.plugin.plugin_manager()
self._gcode_hooks = self._pluginManager.get_hooks("octoprint.comm.protocol.gcode")
self._gcode_hooks = dict(
queuing=self._pluginManager.get_hooks("octoprint.comm.protocol.gcode.queuing"),
queued=self._pluginManager.get_hooks("octoprint.comm.protocol.gcode.queued"),
sending=self._pluginManager.get_hooks("octoprint.comm.protocol.gcode.sending"),
sent=self._pluginManager.get_hooks("octoprint.comm.protocol.gcode.sent")
)
self._printer_action_hooks = self._pluginManager.get_hooks("octoprint.comm.protocol.action")
self._gcodescript_hooks = self._pluginManager.get_hooks("octoprint.comm.protocol.scripts")
self._serial_factory_hooks = self._pluginManager.get_hooks("octoprint.comm.transport.serial.factory")
@ -211,6 +218,8 @@ class MachineCom(object):
self._regex_repetierTempExtr = re.compile("TargetExtr([0-9]+):(%s)" % positiveFloatPattern)
self._regex_repetierTempBed = re.compile("TargetBed:(%s)" % positiveFloatPattern)
self._blocking_commands = ("G28", "G29", "G30", "G32")
# multithreading locks
self._sendNextLock = threading.Lock()
self._sendingLock = threading.RLock()
@ -1416,47 +1425,25 @@ class MachineCom(object):
if self._serial is None:
return
gcode = False
if not self.isStreaming():
for name, hook in self._gcode_hooks.items():
hook_cmd = hook(self, cmd, cmd_type=cmd_type)
# trigger the "queuing" phase only if we are not streaming to sd right now
cmd, cmd_type, gcode = self._process_command_phase("queuing", cmd, cmd_type, gcode=gcode)
if hook_cmd is None:
cmd = None
if cmd is None:
# command is no more, return
return
# hook might have returned (cmd, sendChecksum) or (cmd, sendChecksum, cmd_type), split that
elif isinstance(hook_cmd, tuple):
if len(hook_cmd) == 2:
hook_cmd, cmd_type = hook_cmd
elif len(hook_cmd) == 3:
# legacy hook handler, ignore returned send_checksum
hook_cmd, cmd_type, _ = hook_cmd
if gcode and gcode in gcodeToEvent:
# if this is a gcode bound to an event, trigger that now
eventManager().fire(gcodeToEvent[gcode])
# if hook_cmd is a string, we'll replace cmd with it (it's been rewritten by the hook handler
if isinstance(hook_cmd, basestring):
cmd = hook_cmd
# actually enqueue the command for sending
self._enqueue_for_sending(cmd, command_type=cmd_type)
else:
self._logger.warn("Hook {name} returned unintelligible result, ignoring it: {hook_cmd!r}".format(**locals()))
continue
if cmd is None:
break
# try to parse the cmd and extract the gcode type
if cmd is not None:
gcode = self._regex_command.search(cmd)
if gcode:
gcode = gcode.group(1)
# fire events if necessary
if gcode in gcodeToEvent:
eventManager().fire(gcodeToEvent[gcode])
# send it through the specific handler if it exists
cmd = self._process_command_phase("queue", cmd, gcode=gcode)
if cmd is not None:
self._enqueue_for_sending(cmd, command_type=cmd_type)
if not self.isStreaming():
# trigger the "queued" phase only if we are not streaming to sd right now
self._process_command_phase("queued", cmd, cmd_type, gcode=gcode)
def gcode_command_for_cmd(self, cmd):
"""
@ -1512,23 +1499,28 @@ class MachineCom(object):
break
# fetch command and optional linenumber from queue
command, linenumber, _ = entry
command, linenumber, command_type = entry
# some firmwares (e.g. Smoothie) might support additional in-band communication that will not
# stick to the acknowledgement behaviour of GCODE, so we check here if we have a GCODE command
# at hand here and only clear our clear_to_send flag later if that's the case
gcode_match = self._regex_command.search(command)
is_gcode = gcode_match is not None
if is_gcode:
# trigger "sending" phase
command = self._process_command_phase("sending", command, gcode=gcode_match.group(1))
gcode = self.gcode_command_for_cmd(command)
if linenumber is not None:
# line number predetermined, use that
# line number predetermined - this only happens for resends, so we'll use the number and
# send directly without any processing (since that already took place on the first sending!)
self._doSendWithChecksum(command, linenumber)
else:
if (is_gcode or self._sendChecksumWithUnknownCommands) and (self.isPrinting() or self._alwaysSendChecksum):
# trigger "sending" phase
command, _, gcode = self._process_command_phase("sending", command, command_type, gcode=gcode)
if command is None:
# so no, we are not going to send this, that was a last-minute bail, let's fetch the next item from the queue
continue
# now comes the part where we increase line numbers and send stuff - no turning back now
if (gcode is not None or self._sendChecksumWithUnknownCommands) and (self.isPrinting() or self._alwaysSendChecksum):
linenumber = self._currentLine
self._addToLastLines(command)
self._currentLine += 1
@ -1536,36 +1528,87 @@ class MachineCom(object):
else:
self._doSendWithoutChecksum(command)
# trigger "sent" phase and use up one "ok"
self._process_command_phase("sent", command, command_type, gcode=gcode)
# we only need to use up a clear if the command we just sent was either a gcode command or if we also
# require ack's for unknown commands
use_up_clear = self._unknownCommandsNeedAck
if is_gcode:
# trigger "sent" phase and use up one "ok"
self._process_command_phase("sent", command, gcode=gcode_match.group(1))
if gcode is not None:
use_up_clear = True
# if we need to use up a clear, do that now
if use_up_clear:
self._clear_to_send.clear()
# wait for the next clear
# now we just wait for the next clear and then start again
self._clear_to_send.wait()
except:
self._logger.exception("Caught an exception in the send loop")
self._log("Closing down send loop")
def _process_command_phase(self, phase, command, gcode=None):
def _process_command_phase(self, phase, command, command_type=None, gcode=None):
if phase not in ("queuing", "queued", "sending", "sent"):
return command, command_type, gcode
if gcode is None:
gcode_match = self._regex_command.search(command)
if gcode_match is None:
return command
gcode = self.gcode_command_for_cmd(command)
gcode = gcode.group(1)
# send it through the phase specific handlers provided by plugins
for name, hook in self._gcode_hooks[phase].items():
try:
hook_result = hook(self, phase, command, command_type, gcode if gcode is not False else None)
except:
self._logger.exception("Error while processing hook {name} for phase {phase} and command {command}:".format(**locals()))
else:
command, command_type, gcode = self._handle_command_handler_result(command, command_type, gcode, hook_result)
if command is None:
# hook handler return None as command, so we'll stop here and return a full out None result
return None, None, None
# send it through the specific handler if it exists
gcodeHandler = "_gcode_" + gcode + "_" + phase
if hasattr(self, gcodeHandler):
command = getattr(self, gcodeHandler)(command)
# if it's a gcode command send it through the specific handler if it exists
if gcode is not None:
gcodeHandler = "_gcode_" + gcode + "_" + phase
if hasattr(self, gcodeHandler):
handler_result = getattr(self, gcodeHandler)(command, cmd_type=command_type)
command, command_type, gcode = self._handle_command_handler_result(command, command_type, gcode, handler_result)
return command
# send it through the phase specific command handler if it exists
commandPhaseHandler = "_command_phase_" + phase
if hasattr(self, commandPhaseHandler):
handler_result = getattr(self, commandPhaseHandler)(command, cmd_type=command_type, gcode=gcode if gcode is not False else None)
command, command_type, gcode = self._handle_command_handler_result(command, command_type, gcode, handler_result)
# finally return whatever we resulted on
return command, command_type, gcode
def _handle_command_handler_result(self, command, command_type, gcode, handler_result):
original_tuple = (command, command_type, gcode)
if handler_result is None:
# handler didn't return anything, we'll just continue
return original_tuple
if isinstance(handler_result, basestring):
# handler did return just a string, we'll turn that into a 1-tuple now
handler_result = (handler_result,)
elif not isinstance(handler_result, (tuple, list)):
# handler didn't return an expected result format, we'll just ignore it and continue
return original_tuple
hook_result_length = len(handler_result)
if hook_result_length == 1:
# handler returned just the command
command, = handler_result
elif hook_result_length == 2:
# handler returned command and command_type
command, command_type = handler_result
else:
# handler returned a tuple of an unexpected length
return original_tuple
gcode = self.gcode_command_for_cmd(command)
return command, command_type, gcode
##~~ actual sending via serial
@ -1594,13 +1637,12 @@ class MachineCom(object):
##~~ command handlers
def _gcode_T_sent(self, cmd):
def _gcode_T_sent(self, cmd, cmd_type=None):
toolMatch = self._regex_paramTInt.search(cmd)
if toolMatch:
self._currentTool = int(toolMatch.group(1))
return cmd
def _gcode_G0_sent(self, cmd):
def _gcode_G0_sent(self, cmd, cmd_type=None):
if 'Z' in cmd:
match = self._regex_paramZFloat.search(cmd)
if match:
@ -1611,15 +1653,14 @@ class MachineCom(object):
self._callback.on_comm_z_change(z)
except ValueError:
pass
return cmd
_gcode_G1_sent = _gcode_G0_sent
def _gcode_M0_queue(self, cmd):
def _gcode_M0_queuing(self, cmd, cmd_type=None):
self.setPause(True)
return None # Don't send the M0 or M1 to the machine, as M0 and M1 are handled as an LCD menu pause.
_gcode_M1_queued = _gcode_M0_queue
return None, # Don't send the M0 or M1 to the machine, as M0 and M1 are handled as an LCD menu pause.
_gcode_M1_queuing = _gcode_M0_queuing
def _gcode_M104_sent(self, cmd):
def _gcode_M104_sent(self, cmd, cmd_type=None):
toolNum = self._currentTool
toolMatch = self._regex_paramTInt.search(cmd)
if toolMatch:
@ -1635,9 +1676,8 @@ class MachineCom(object):
self._temp[toolNum] = (None, target)
except ValueError:
pass
return cmd
def _gcode_M140_sent(self, cmd):
def _gcode_M140_sent(self, cmd, cmd_type=None):
match = self._regex_paramSInt.search(cmd)
if match:
try:
@ -1649,21 +1689,20 @@ class MachineCom(object):
self._bedTemp = (None, target)
except ValueError:
pass
return cmd
def _gcode_M109_sent(self, cmd):
def _gcode_M109_sent(self, cmd, cmd_type=None):
self._heatupWaitStartTime = time.time()
self._blocking_command = True
self._heating = True
return self._gcode_M104_sent(cmd)
self._gcode_M104_sent(cmd, cmd_type)
def _gcode_M190_sent(self, cmd):
def _gcode_M190_sent(self, cmd, cmd_type=None):
self._heatupWaitStartTime = time.time()
self._blocking_command = True
self._heating = True
return self._gcode_M140_sent(cmd)
self._gcode_M140_sent(cmd, cmd_type)
def _gcode_M110_sending(self, cmd):
def _gcode_M110_sending(self, cmd, cmd_type=None):
newLineNumber = None
match = self._regex_paramNInt.search(cmd)
if match:
@ -1681,13 +1720,10 @@ class MachineCom(object):
self._lastLines.clear()
self._resendDelta = None
return cmd
def _gcode_M112_queue(self, cmd): # It's an emergency what todo? Canceling the print should be the minimum
def _gcode_M112_queuing(self, cmd, cmd_type=None): # It's an emergency what todo? Canceling the print should be the minimum
self.cancelPrint()
return cmd
def _gcode_G4_sent(self, cmd):
def _gcode_G4_sent(self, cmd, cmd_type=None):
# we are intending to dwell for a period of time, increase the timeout to match
cmd = cmd.upper()
p_idx = cmd.find('P')
@ -1701,14 +1737,12 @@ class MachineCom(object):
_timeout = float(cmd[s_idx+1:])
self._timeout = get_new_timeout("communication") + _timeout
self._blocking_command = True
return cmd
def _gcode_G28_sending(self, cmd):
self._blocking_command = True
return cmd
_gcode_G29_sending = _gcode_G28_sending
_gcode_G30_sending = _gcode_G28_sending
_gcode_G32_sending = _gcode_G28_sending
##~~ command phase handlers
def _command_phase_sending(self, cmd, cmd_type=None, gcode=None):
if gcode is not None and gcode in self._blocking_commands:
self._blocking_command = True
### MachineCom callback ################################################################################################