diff --git a/CHANGELOG.md b/CHANGELOG.md index 67ecc184..fe57f3e8 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,6 +1,6 @@ # OctoPrint Changelog -## 1.2.0 (Unreleased) +## 1.2.0 (2015-06-25) ### Note for Upgraders @@ -11,27 +11,44 @@ ### New Features +* OctoPrint now has a [plugin system](http://docs.octoprint.org/en/master/plugins/index.html) which allows extending its + core functionality. + * Plugins may be installed through the new and bundled [Plugin Manager Plugin](https://github.com/foosel/OctoPrint/wiki/Plugin:-Plugin-Manager) + available in OctoPrint's settings. This Plugin Manager also allows browsing and easy installation of plugins + registered on the official [OctoPrint Plugin Repository](http://plugins.octoprint.org). + * For interested developers there is a [tutorial available in the documentation](http://docs.octoprint.org/en/master/plugins/gettingstarted.html) + and also a [cookiecutter template](https://github.com/OctoPrint/cookiecutter-octoprint-plugin) to quickly bootstrap + new plugins. * Added internationalization of UI. Translations of OctoPrint are being crowd sourced via [Transifex](https://www.transifex.com/projects/p/octoprint/). Language Packs for both the core application as well as installed plugins can be uploaded through a new management - dialog in Settings > Appearance > Language Packs. + dialog in Settings > Appearance > Language Packs. A translation into German is included, further language packs + will soon be made available. +* Printer Profiles: Printer properties like print volume, extruder offsets etc are now managed via Printer Profiles. A + connection to a printer will always have a printer profile associated. +* File management now supports STL files as first class citizens (including UI adjustments to allow management of + uploaded STL files including removal and reslicing) and also allows folders (not yet supported by UI). STL files + can be downloaded like GCODE files. +* Slicing has been greatly improved: + * It now allows for a definition of slicing profiles to use for slicing plus overrides which can be defined per slicing + job (defining overrides is not yet part of the UI but it's on the roadmap). + * A new slicing dialog has been added which allows (re-)slicing uploaded STL files (which are now displayed in the file list + as well). The slicing profile and printer profile to use can be specified here as well as the file name to which to + slice to and the action to take after slicing has been completed (none, selecting the sliced GCODE for printing or + starting to print it directly) + * The slicing API allows positioning the model to slice on the print bed (Note: this is not yet available in the UI). + * Slicers themselves are integrated into the system via ``SlicingPlugins``. + * Bundled [Cura Plugin](https://github.com/foosel/OctoPrint/wiki/Plugin:-Cura) allows slicing through CuraEngine up + to and including 15.04. Existing Cura slicing profiles can be imported through the web interface. * New file list: Pagination is gone, no more (mobile incompatible) pop overs, instead scrollable and with instant search * You can now define a folder (default: `~/.octoprint/watched`) to be watched for newly added GCODE (or -- if slicing support is enabled -- STL) files to automatically add. -* OctoPrint now has a [plugin system](http://docs.octoprint.org/en/devel/plugins/index.html) which allows extending its - core functionality. Plugins may be installed through the new Plugin Manager available in OctoPrint's settings. This - Plugin Manager also allows browsing and easy installation of plugins registered on the official - [OctoPrint Plugin Repository](http://plugins.octoprint.org). For interested developers there is a - [tutorial available in the documentation](http://docs.octoprint.org/en/master/plugins/gettingstarted.html) and also a - [cookiecutter template](https://github.com/OctoPrint/cookiecutter-octoprint-plugin) to quickly bootstrap new plugins. -* New type of API key: [App Session Keys](http://docs.octoprint.org/en/devel/api/apps.html) for trusted applications -* Printer Profiles: Printer properties like print volume, extruder offsets etc are now managed via Printer Profiles. A - connection to a printer will always have a printer profile associated. +* New type of API key: [App Session Keys](http://docs.octoprint.org/en/master/api/apps.html) for trusted applications * OctoPrint now supports `action:...` commands received via debug messages (`// action:...`) from the printer. Currently supported are - `action:pause`: Pauses the current job in OctoPrint - `action:resume`: Resumes the current job in OctoPrint - `action:disconnect`: Disconnects OctoPrint from the printer - Plugins can add supported commands by [hooking](http://docs.octoprint.org/en/devel/plugins/hooks.html) into the + Plugins can add supported commands by [hooking](http://docs.octoprint.org/en/master/plugins/hooks.html) into the ``octoprint.comm.protocol.action`` hook * Mousing over the webcam image in the control tab enables key control mode, allowing you to quickly move the axis of your printer with your computer's keyboard ([#610](https://github.com/foosel/OctoPrint/pull/610)): @@ -45,9 +62,11 @@ * Custom controls now support a row layout * Users can now define custom GCODE scripts to run upon starting/pausing/resuming/success/failure of a print or for custom controls ([#457](https://github.com/foosel/OctoPrint/issues/457), [#347](https://github.com/foosel/OctoPrint/issues/347)) +* Bundled [Discovery Plugin](https://github.com/foosel/OctoPrint/wiki/Plugin:-Discovery) allows discovery of OctoPrint + instances via SSDP/UPNP and optionally also via ZeroConf/Bonjour/Avahi. * Bundled [Software Update Plugin](https://github.com/foosel/OctoPrint/wiki/Plugin:-Software-Update) takes care of notifying about new OctoPrint releases and also allows updating if configured as such. Plugins may register themselves with the - update notification and application process through a new hook ["octoprint.plugin.softwareupdate.check_config"](). + update notification and application process through a new hook ["octoprint.plugin.softwareupdate.check_config"](https://github.com/foosel/OctoPrint/wiki/Plugin:-Software-Update#octoprintpluginsoftwareupdatecheck_config). ### Improvements @@ -60,19 +79,7 @@ * Better error reporting for timelapse rendering and system commands * Custom control can now be defined so that they show a Confirm dialog with configurable text before executing ([#532](https://github.com/foosel/OctoPrint/issues/532) and [#590](https://github.com/foosel/OctoPrint/pull/590)) -* Slicing has been greatly improved: - * It now allows for a definition of slicing profiles to use for slicing plus overrides which can be defined per slicing - job (defining overrides is not yet part of the UI but it's on the roadmap). - * A new slicing dialog has been added which allows (re-)slicing uploaded STL files (which are now displayed in the file list - as well). This dialog also allows specifying which action to take after slicing has been completed (none, selecting the - sliced GCODE for printing or starting to print it directly) - * Slicers themselves are integrated into the system via ``SlicingPlugins``. - * The "Slicing done" notification is now colored green ([#558](https://github.com/foosel/OctoPrint/issues/558)). - * The slicing API allows positioning the model to slice on the print bed (Note: this is not yet available in the UI). -* File management now supports STL files as first class citizens (including UI adjustments to allow management of - uploaded STL files including removal and reslicing) and also allows folders (not yet supported by UI). STL files - can be downloaded like GCODE files. -* Also interpret lines starting with "!!" as errors +* Serial communication: Also interpret lines starting with "!!" as errors * Added deletion of pyc files to the `python setup.py clean` command * Settings now show a QRCode for the API Key ([#637](https://github.com/foosel/OctoPrint/pull/637)) * Username in UI is no longer enclosed in scare quotes ([#595](https://github.com/foosel/OctoPrint/pull/595)) @@ -90,13 +97,14 @@ * The server now tracks the modification date of the configuration file and reloads it prior to saving the config if it has been changed during runtime by external editing, hence no config settings added manually while the server was running should be overwritten anymore. -* Automatically hard-reload the UI if upon reconnecting to the server a new version is detected. +* Display a "Please Reload" overlay when a new server version or a change in installed plugins is detected after a + reconnect to the server. * Better handling of errors on the websocket - no more logging of the full stack trace to the log, only a warning message for now. * Daemonized OctoPrint now cleans up its pidfile when receiving a TERM signal ([#711](https://github.com/foosel/OctoPrint/issues/711)) * Added serial types for OpenBSD ([#551](https://github.com/foosel/OctoPrint/pull/551)) * Improved behaviour of terminal: - * Disabling autoscrolling now also stops cutting of the log while it's enabled, effectively preventing log lines from + * Disabling autoscrolling now also stops cutting off the log while it's enabled, effectively preventing log lines from being modified at all ([#735](https://github.com/foosel/OctoPrint/issues/735)) * Applying filters displays ``[...]`` where lines where removed and doesn't cause scrolling on filtered lines anymore ([#286](https://github.com/foosel/OctoPrint/issues/286)) @@ -123,7 +131,6 @@ from each other (see also [#823](https://github.com/foosel/OctoPrint/pull/823)) * Renamed "Temperature Timeout" and "SD Status Timeout" in Settings to "Temperature Interval" and "SD Status Interval" to better reflect what those values are actually used for. -* Better behaviour of the settings dialog on mobile devices. * Added support for rectangular printer beds with the origin in the center ([#682](https://github.com/foosel/OctoPrint/issues/682) and [#852](https://github.com/foosel/OctoPrint/pull/852)). Printer profiles now contain a new settings ``volume.origin`` which can either be ``lowerleft`` or ``center``. For circular beds only ``center`` is supported. @@ -137,6 +144,24 @@ * Stop websocket connections from multiplying ([#888](https://github.com/foosel/OctoPrint/pull/888)). * New setting to rotate webcam by 90° counter clockwise ([#895](https://github.com/foosel/OctoPrint/issues/895) and [#906](https://github.com/foosel/OctoPrint/pull/906)) +* System commands now be set to a) run asynchronized by setting their `async` property to `true` and b) to ignore their + result by setting their `ignore` property to `true`. +* Various improvements of newly introduced features over the course of development: + * File management: The new implementation will migrate metadata from the old one upon first startup after upgrade from + version 1.1.x to 1.2.x. That should speed up initial startup. + * File management: GCODE Analysis backlog processing has been throttled to not take up too many resources on system + startup. Freshly uploaded files should still be analyzed at full speed. + * Plugins: SettingsPlugins may track versions of configuration format stored in `config.yaml`, including a custom + migration method getting called when a mismatch between the currently stored configuration format version and the one + reported by the plugin as current is detected. + * Plugins: Plugins may now have a folder for plugin related data whose path can be retrieved from the plugin itself + via its new method [`get_plugin_data_folder`](http://docs.octoprint.org/en/master/modules/plugin.html#octoprint.plugin.types.OctoPrintPlugin.get_plugin_data_folder). + * Plugin Manager: Don't allow plugin management actions (like installing/uninstalling or enabling/disabling) while the + printer is printing (see also unreproduced issue [#936](https://github.com/foosel/OctoPrint/issues/936)). + * Plugin Manager: More options to try to match up installed plugin packages with discovered plugins. + * Plugin Manager: Display a more friendly message if after the installation of a plugin it could not be correctly + identifier. + * Software Update: Enforce refreshing of available updates after any changes in enabled plugins. ### Bug Fixes @@ -201,8 +226,14 @@ downloaded from Github. * [#931](https://github.com/foosel/OctoPrint/issues/931) - Adjusted `octoprint_setuptools` to be compatible to older versions of setuptools potentially site-wide installed on hosts. - * Plugin Manager supports installing plugins that need `--process-dependency-links` as `pip` argument. This might be - necessary when plugins depend on (patched) library versions that are not yet released on PyPI + * [#942](https://github.com/foosel/OctoPrint/issues/942) - Settings can now be saved again after installing a new + plugin. Plugins must not use `super` anymore to call parent implementation of `SettingsPlugin.on_settings_save` but + should instead switch to `SettingsPlugin.on_settings_save(self, ...)`. Settings API will capture related + `TypeErrors` and log a big warning to the log file indicating which plugin caused the problem and needs to be + updated. Also updated all bundled plugins accordingly. + * Software Update: Don't persist more check data than necessary in the configuration. Solves an issue where persisted + information overrode updated check configuration reported by plugins, leading to a "an update is available" loop. + An auto-migration function was added that should remove the redundant data. * Various fixes without tickets: * GCODE viewer now doesn't stumble over completely extrusionless GCODE files * Do not deliver the API key on settings API unless user has admin rights @@ -220,7 +251,13 @@ "Operational" state. * Log cancelled prints only once (thanks to @imrahil for the headsup) -([Commits](https://github.com/foosel/OctoPrint/compare/1.1.2...master)) +### More information + + * [Commits](https://github.com/foosel/OctoPrint/compare/1.1.2...1.2.0) + * Release Candidates: + * [RC1](https://github.com/foosel/OctoPrint/releases/tag/1.2.0-rc1) + * [RC2](https://github.com/foosel/OctoPrint/releases/tag/1.2.0-rc2) + * [RC3](https://github.com/foosel/OctoPrint/releases/tag/1.2.0-rc3) ## 1.1.2 (2015-03-23) diff --git a/docs/plugins/concepts.rst b/docs/plugins/concepts.rst index ecc8ecbb..42b66d37 100644 --- a/docs/plugins/concepts.rst +++ b/docs/plugins/concepts.rst @@ -257,7 +257,7 @@ them as (hopefully) documented. :emphasize-lines: 6-8,20 :caption: Excerpt from the Growl Plugin showing utilization of the helpers published by the Discovery Plugin. :name: sec-plugin-concepts-helpers-example-usage - + def on_after_startup(self): host = self._settings.get(["hostname"]) port = self._settings.getInt(["port"]) @@ -270,7 +270,7 @@ them as (hopefully) documented. self.growl, _ = self._register_growl(host, port, password=password) # ... - + def on_api_get(self, request): if not self.zeroconf_browse: return flask.jsonify(dict( @@ -302,6 +302,10 @@ An overview of these properties follows. ``self._basefolder`` The plugin's base folder where it's installed. Can be used to refer to files relative to the plugin's installation location, e.g. included scripts, templates or assets. +``self._datafolder`` + The plugin's additional data folder path. Can be used to store additional files needed for the plugin's operation (cache, + data files etc). Plugins should not access this property directly but instead utilize :func:`~octoprint.plugin.types.OctoPrintPlugin.get_plugin_data_folder` + which will make sure the path actually does exist and if not create it before returning it. ``self._logger`` A `python logger instance `_ logging to the log target ``octoprint.plugin.``. @@ -340,4 +344,4 @@ Lifecycle .. image:: ../images/plugins_lifecycle.png :align: center - :alt: The lifecycle of OctoPrint plugins. \ No newline at end of file + :alt: The lifecycle of OctoPrint plugins. diff --git a/src/octoprint/plugin/__init__.py b/src/octoprint/plugin/__init__.py index ce7224b3..21976d08 100644 --- a/src/octoprint/plugin/__init__.py +++ b/src/octoprint/plugin/__init__.py @@ -426,12 +426,10 @@ class PluginSettings(object): filename += ".log" return os.path.join(self.settings.getBaseFolder("logs"), filename) + @deprecated("PluginSettings.get_plugin_data_folder has been replaced by OctoPrintPlugin.get_plugin_data_folder", + includedoc="Replaced by :func:`~octoprint.plugin.types.OctoPrintPlugin.get_plugin_data_folder`", + since="1.2.0") def get_plugin_data_folder(self): - """ - Retrieves the path to a data folder specifically for the plugin. - - Plugins may use this for storing additional data. - """ path = os.path.join(self.settings.getBaseFolder("data"), self.plugin_key) if not os.path.isdir(path): os.makedirs(path) diff --git a/src/octoprint/plugin/types.py b/src/octoprint/plugin/types.py index 578e117a..ad74364c 100644 --- a/src/octoprint/plugin/types.py +++ b/src/octoprint/plugin/types.py @@ -69,9 +69,32 @@ class OctoPrintPlugin(Plugin): The :class:`~octoprint.server.LifecycleManager` instance. Injected by the plugin core system upon initialization of the implementation. + + .. attribute:: _data_folder + + Path to the data folder for the plugin to use for any data it might have to persist. Should always be accessed + through :meth:`get_plugin_data_folder` since that function will also ensure that the data folder actually exists + and if not creating it before returning it. Injected by the plugin core system upon initialization of the + implementation. + + .. automethod:: get_plugin_data_folder """ - pass + def get_plugin_data_folder(self): + """ + Retrieves the path to a data folder specifically for the plugin, ensuring it exists and if not creating it + before returning it. + + Plugins may use this folder for storing additional data they need for their operation. + """ + if self._data_folder is None: + raise RuntimeError("self._plugin_data_folder is None, has the plugin been initialized yet?") + + import os + if not os.path.isdir(self._data_folder): + os.makedirs(self._data_folder) + return self._data_folder + class ReloadNeedingPlugin(Plugin): pass diff --git a/src/octoprint/plugins/pluginmanager/__init__.py b/src/octoprint/plugins/pluginmanager/__init__.py index e59de75e..152d9b41 100644 --- a/src/octoprint/plugins/pluginmanager/__init__.py +++ b/src/octoprint/plugins/pluginmanager/__init__.py @@ -12,6 +12,7 @@ import octoprint.plugin.core from octoprint.settings import valid_boolean_trues from octoprint.server.util.flask import restricted_access from octoprint.server import admin_permission +from octoprint.util.pip import PipCaller, UnknownPip from flask import jsonify, make_response from flask.ext.babel import gettext @@ -22,6 +23,7 @@ import sys import requests import re import os +import pkg_resources class PluginManagerPlugin(octoprint.plugin.SimpleApiPlugin, octoprint.plugin.TemplatePlugin, @@ -36,6 +38,8 @@ class PluginManagerPlugin(octoprint.plugin.SimpleApiPlugin, self._pending_install = set() self._pending_uninstall = set() + self._pip_caller = None + self._repository_available = False self._repository_plugins = [] self._repository_cache_path = None @@ -43,9 +47,14 @@ class PluginManagerPlugin(octoprint.plugin.SimpleApiPlugin, def initialize(self): self._console_logger = logging.getLogger("octoprint.plugins.pluginmanager.console") - self._repository_cache_path = os.path.join(self._settings.get_plugin_data_folder(), "plugins.json") + self._repository_cache_path = os.path.join(self.get_plugin_data_folder(), "plugins.json") self._repository_cache_ttl = self._settings.get_int(["repository_ttl"]) * 60 + self._pip_caller = PipCaller(configured=self._settings.get(["pip"])) + self._pip_caller.on_log_call = self._log_call + self._pip_caller.on_log_stdout = self._log_stdout + self._pip_caller.on_log_stderr = self._log_stderr + ##~~ StartupPlugin def on_startup(self, host, port): @@ -65,13 +74,13 @@ class PluginManagerPlugin(octoprint.plugin.SimpleApiPlugin, return dict( repository="http://plugins.octoprint.org/plugins.json", repository_ttl=24*60, - pip=None, - dependency_links=False + pip=None ) def on_settings_save(self, data): octoprint.plugin.SettingsPlugin.on_settings_save(self, data) self._repository_cache_ttl = self._settings.get_int(["repository_ttl"]) * 60 + self._pip_caller.refresh = True ##~~ AssetPlugin @@ -160,7 +169,6 @@ class PluginManagerPlugin(octoprint.plugin.SimpleApiPlugin, plugin_name = data["plugin"] if "plugin" in data else None return self.command_install(url=url, force="force" in data and data["force"] in valid_boolean_trues, - dependency_links="dependency_links" in data and data["dependency_links"] in valid_boolean_trues, reinstall=plugin_name) elif command == "uninstall": @@ -179,7 +187,7 @@ class PluginManagerPlugin(octoprint.plugin.SimpleApiPlugin, plugin = self._plugin_manager.plugins[plugin_name] return self.command_toggle(plugin, command) - def command_install(self, url=None, path=None, force=False, reinstall=None, dependency_links=False): + def command_install(self, url=None, path=None, force=False, reinstall=None): if url is not None: pip_args = ["install", sarge.shell_quote(url)] elif path is not None: @@ -187,9 +195,6 @@ class PluginManagerPlugin(octoprint.plugin.SimpleApiPlugin, else: raise ValueError("Either url or path must be provided") - if dependency_links or self._settings.get_boolean(["dependency_links"]): - pip_args.append("--process-dependency-links") - all_plugins_before = self._plugin_manager.find_plugins() success_string = "Successfully installed" @@ -396,74 +401,12 @@ class PluginManagerPlugin(octoprint.plugin.SimpleApiPlugin, self._plugin_manager.send_plugin_message(self._identifier, notification) def _call_pip(self, args): - pip_command = self._settings.get(["pip"]) - if pip_command is None: - import os - python_command = sys.executable - binary_dir = os.path.dirname(python_command) + if self._pip_caller is None or not self._pip_caller.available: + raise RuntimeError(u"No pip available, can't operate".format(**locals())) + return self._pip_caller.execute(*args) - pip_command = os.path.join(binary_dir, "pip") - if sys.platform == "win32": - # Windows is a bit special... first of all the file will be called pip.exe, not just pip, and secondly - # for a non-virtualenv install (e.g. global install) the pip binary will not be located in the - # same folder as python.exe, but in a subfolder Scripts, e.g. - # - # C:\Python2.7\ - # |- python.exe - # `- Scripts - # `- pip.exe - - # virtual env? - pip_command = os.path.join(binary_dir, "pip.exe") - - if not os.path.isfile(pip_command): - # nope, let's try the Scripts folder then - scripts_dir = os.path.join(binary_dir, "Scripts") - if os.path.isdir(scripts_dir): - pip_command = os.path.join(scripts_dir, "pip.exe") - - if not os.path.isfile(pip_command) or not os.access(pip_command, os.X_OK): - raise RuntimeError(u"No pip path configured and {pip_command} does not exist or is not executable, can't install".format(**locals())) - - command = [pip_command] + args - - self._logger.debug(u"Calling: {}".format(" ".join(command))) - - p = sarge.run(" ".join(command), shell=True, async=True, stdout=sarge.Capture(), stderr=sarge.Capture()) - p.wait_events() - - all_stdout = [] - all_stderr = [] - try: - while p.returncode is None: - line = p.stderr.readline(timeout=0.5) - if line: - self._log_stderr(line) - all_stderr.append(line) - - line = p.stdout.readline(timeout=0.5) - if line: - self._log_stdout(line) - all_stdout.append(line) - - p.commands[0].poll() - - finally: - p.close() - - stderr = p.stderr.text - if stderr: - split_lines = stderr.split("\n") - self._log_stderr(*split_lines) - all_stderr += split_lines - - stdout = p.stdout.text - if stdout: - split_lines = stdout.split("\n") - self._log_stdout(*split_lines) - all_stdout += split_lines - - return p.returncode, all_stdout, all_stderr + def _log_call(self, *lines): + self._log(lines, prefix=" ", stream="call") def _log_stdout(self, *lines): self._log(lines, prefix=">", stream="stdout") @@ -560,9 +503,6 @@ class PluginManagerPlugin(octoprint.plugin.SimpleApiPlugin, def map_repository_entry(entry): result = dict(entry) - if not "follow_dependency_links" in result: - result["follow_dependency_links"] = False - result["is_compatible"] = dict( octoprint=True, os=True diff --git a/src/octoprint/plugins/pluginmanager/static/css/pluginmanager.css b/src/octoprint/plugins/pluginmanager/static/css/pluginmanager.css index 76eea944..b1cbc9af 100644 --- a/src/octoprint/plugins/pluginmanager/static/css/pluginmanager.css +++ b/src/octoprint/plugins/pluginmanager/static/css/pluginmanager.css @@ -1 +1 @@ -table th.settings_plugin_plugin_manager_plugins_name,table td.settings_plugin_plugin_manager_plugins_name{text-overflow:ellipsis;text-align:left}table th.settings_plugin_plugin_manager_plugins_actions,table td.settings_plugin_plugin_manager_plugins_actions{text-align:center;width:80px}table th.settings_plugin_plugin_manager_plugins_actions a,table td.settings_plugin_plugin_manager_plugins_actions a{text-decoration:none;color:#000}table th.settings_plugin_plugin_manager_plugins_actions a.disabled,table td.settings_plugin_plugin_manager_plugins_actions a.disabled{color:#ccc;cursor:default}#settings_plugin_pluginmanager_repositorydialog .slimScrollDiv{margin-bottom:20px}#settings_plugin_pluginmanager_repositorydialog h4{position:relative}#settings_plugin_pluginmanager_repositorydialog h4 a.dropdown-toggle{color:inherit;text-decoration:none;font-size:14px}#settings_plugin_pluginmanager_repositorydialog h4 ul.dropdown-menu{font-size:14px}#settings_plugin_pluginmanager_repositorydialog .form-search{text-align:center;margin-bottom:5px!important}#settings_plugin_pluginmanager_repositorydialog .form-inline{padding:5px;padding-right:10px;margin-bottom:0}#settings_plugin_pluginmanager_repositorydialog .form-inline .help-block{margin-bottom:0;font-size:85%}#settings_plugin_pluginmanager_repositorydialog #settings_plugin_pluginmanager_repositorydialog_unavailable{overflow:hidden;width:100%;height:300px;background-image:url("../img/repo_unavailable.png");text-align:center;display:table}#settings_plugin_pluginmanager_repositorydialog #settings_plugin_pluginmanager_repositorydialog_unavailable div{display:table-cell;vertical-align:middle}#settings_plugin_pluginmanager_repositorydialog #settings_plugin_pluginmanager_repositorydialog_list{overflow:hidden;width:auto;height:300px}#settings_plugin_pluginmanager_repositorydialog #settings_plugin_pluginmanager_repositorydialog_list .entry{border-bottom:1px solid #ddd;padding:5px;padding-right:10px} \ No newline at end of file +table th.settings_plugin_plugin_manager_plugins_name,table td.settings_plugin_plugin_manager_plugins_name{text-overflow:ellipsis;text-align:left}table th.settings_plugin_plugin_manager_plugins_actions,table td.settings_plugin_plugin_manager_plugins_actions{text-align:center;width:80px}table th.settings_plugin_plugin_manager_plugins_actions a,table td.settings_plugin_plugin_manager_plugins_actions a{text-decoration:none;color:#000}table th.settings_plugin_plugin_manager_plugins_actions a.disabled,table td.settings_plugin_plugin_manager_plugins_actions a.disabled{color:#ccc;cursor:default}#settings_plugin_pluginmanager_repositorydialog .slimScrollDiv{margin-bottom:20px}#settings_plugin_pluginmanager_repositorydialog h4{position:relative}#settings_plugin_pluginmanager_repositorydialog h4 a.dropdown-toggle{color:inherit;text-decoration:none;font-size:14px}#settings_plugin_pluginmanager_repositorydialog h4 ul.dropdown-menu{font-size:14px}#settings_plugin_pluginmanager_repositorydialog .form-search{text-align:center;margin-bottom:5px!important}#settings_plugin_pluginmanager_repositorydialog .form-inline{padding:5px;padding-right:10px;margin-bottom:0}#settings_plugin_pluginmanager_repositorydialog .form-inline .help-block{margin-bottom:0;font-size:85%}#settings_plugin_pluginmanager_repositorydialog #settings_plugin_pluginmanager_repositorydialog_unavailable{overflow:hidden;width:100%;height:300px;background-image:url("../img/repo_unavailable.png");text-align:center;display:table}#settings_plugin_pluginmanager_repositorydialog #settings_plugin_pluginmanager_repositorydialog_unavailable div{display:table-cell;vertical-align:middle}#settings_plugin_pluginmanager_repositorydialog #settings_plugin_pluginmanager_repositorydialog_list{overflow:hidden;width:auto;height:300px}#settings_plugin_pluginmanager_repositorydialog #settings_plugin_pluginmanager_repositorydialog_list .entry{border-bottom:1px solid #ddd;padding:5px;padding-right:10px}#settings_plugin_pluginmanager_workingdialog_output .message{font-weight:bold}#settings_plugin_pluginmanager_workingdialog_output .stdout{color:#333}#settings_plugin_pluginmanager_workingdialog_output .stderr{color:#900}#settings_plugin_pluginmanager_workingdialog_output .call{color:#009} \ No newline at end of file diff --git a/src/octoprint/plugins/pluginmanager/static/js/pluginmanager.js b/src/octoprint/plugins/pluginmanager/static/js/pluginmanager.js index f7508851..04aa302c 100644 --- a/src/octoprint/plugins/pluginmanager/static/js/pluginmanager.js +++ b/src/octoprint/plugins/pluginmanager/static/js/pluginmanager.js @@ -70,8 +70,6 @@ $(function() { self.loglines = ko.observableArray([]); self.installedPlugins = ko.observableArray([]); - self.followDependencyLinks = ko.observable(false); - self.working = ko.observable(false); self.workingTitle = ko.observable(); self.workingDialog = undefined; @@ -127,9 +125,6 @@ $(function() { self.uploadButton.unbind("click"); self.uploadButton.bind("click", function() { self._markWorking(gettext("Installing plugin..."), gettext("Installing plugin from uploaded archive...")); - data.formData = { - dependency_links: self.followDependencyLinks() - }; data.submit(); return false; }); @@ -243,13 +238,13 @@ $(function() { } if (self.installed(data)) { - self.installPlugin(data.archive, data.title, data.id, data.follow_dependency_links || self.followDependencyLinks()); + self.installPlugin(data.archive, data.title, data.id); } else { - self.installPlugin(data.archive, data.title, undefined, data.follow_dependency_links || self.followDependencyLinks()); + self.installPlugin(data.archive, data.title, undefined); } }; - self.installPlugin = function(url, name, reinstall, followDependencyLinks) { + self.installPlugin = function(url, name, reinstall) { if (!self.loginState.isAdmin()) { return; } @@ -263,10 +258,6 @@ $(function() { } if (!url) return; - if (followDependencyLinks === undefined) { - followDependencyLinks = self.followDependencyLinks(); - } - var workTitle, workText; if (!reinstall) { workTitle = gettext("Installing plugin..."); @@ -282,7 +273,7 @@ $(function() { self._markWorking(workTitle, workText); var command = "install"; - var payload = {url: url, dependency_links: followDependencyLinks}; + var payload = {url: url}; if (reinstall) { payload["plugin"] = reinstall; payload["force"] = true; diff --git a/src/octoprint/plugins/pluginmanager/static/less/pluginmanager.less b/src/octoprint/plugins/pluginmanager/static/less/pluginmanager.less index 6f21d187..a14c2f6b 100644 --- a/src/octoprint/plugins/pluginmanager/static/less/pluginmanager.less +++ b/src/octoprint/plugins/pluginmanager/static/less/pluginmanager.less @@ -84,3 +84,20 @@ table { } } +#settings_plugin_pluginmanager_workingdialog_output { + .message { + font-weight: bold; + } + + .stdout { + color: #333333; + } + + .stderr { + color: #990000; + } + + .call { + color: #000099; + } +} diff --git a/src/octoprint/plugins/pluginmanager/templates/pluginmanager_settings.jinja2 b/src/octoprint/plugins/pluginmanager/templates/pluginmanager_settings.jinja2 index 8cffb17f..bc05691e 100644 --- a/src/octoprint/plugins/pluginmanager/templates/pluginmanager_settings.jinja2 +++ b/src/octoprint/plugins/pluginmanager/templates/pluginmanager_settings.jinja2 @@ -53,7 +53,7 @@

-

+

@@ -147,21 +147,6 @@
{{ _('This does not look like a valid plugin archive. Valid plugin archives should be either zip files or tarballs and have the extension ".zip", ".tar.gz", ".tgz" or ".tar"') }} - -
-
{{ _('Advanced options') }}
-
-
-
-
- -
-
-
-
-
diff --git a/src/octoprint/plugins/softwareupdate/__init__.py b/src/octoprint/plugins/softwareupdate/__init__.py index 1efdb064..e6972737 100644 --- a/src/octoprint/plugins/softwareupdate/__init__.py +++ b/src/octoprint/plugins/softwareupdate/__init__.py @@ -42,7 +42,7 @@ class SoftwareUpdatePlugin(octoprint.plugin.BlueprintPlugin, def initialize(self): self._version_cache_ttl = self._settings.get_int(["cache_ttl"]) * 60 - self._version_cache_path = os.path.join(self._settings.get_plugin_data_folder(), "versioncache.yaml") + self._version_cache_path = os.path.join(self.get_plugin_data_folder(), "versioncache.yaml") self._load_version_cache() def refresh_checks(name, plugin): diff --git a/src/octoprint/plugins/softwareupdate/updaters/pip.py b/src/octoprint/plugins/softwareupdate/updaters/pip.py index c6f8e590..36f52812 100644 --- a/src/octoprint/plugins/softwareupdate/updaters/pip.py +++ b/src/octoprint/plugins/softwareupdate/updaters/pip.py @@ -7,114 +7,68 @@ __copyright__ = "Copyright (C) 2014 The OctoPrint Project - Released under terms import logging -import sarge -import sys + +from octoprint.util.pip import PipCaller, UnknownPip logger = logging.getLogger("octoprint.plugins.softwareupdate.updaters.pip") console_logger = logging.getLogger("octoprint.plugins.softwareupdate.updaters.pip.console") +_pip_callers = dict() + def can_perform_update(target, check): - return "pip" in check + pip_caller = _get_pip_caller(command=check["pip_command"] if "pip_command" in check else None) + return "pip" in check and pip_caller is not None and pip_caller.available + +def _get_pip_caller(command=None): + key = command + if command is None: + key = "__default" + + if not key in _pip_callers: + try: + _pip_callers[key] = PipCaller(configured=command) + _pip_callers[key].on_log_call = _log_call + _pip_callers[key].on_log_stdout = _log_stdout + _pip_callers[key].on_log_stderr = _log_stderr + except UnknownPip: + _pip_callers[key] = None + + return _pip_callers[key] def perform_update(target, check, target_version): pip_command = None if "pip_command" in check: pip_command = check["pip_command"] + pip_caller = _get_pip_caller(command=pip_command) + if pip_caller is None: + raise RuntimeError("Can't run pip") + install_arg = check["pip"].format(target_version=target_version) logger.debug("Target: %s, executing pip install %s" % (target, install_arg)) pip_args = ["install", check["pip"].format(target_version=target_version, target=target_version)] - if "dependency_links" in check and check["dependency_links"]: - pip_args += ["--process-dependency-links"] + pip_caller.execute(*pip_args) - _call_pip(pip_args, pip_command=pip_command) - - logger.debug("Target. %s, executing pip install %s --ignore-reinstalled --force-reinstall --no-deps" % (target, install_arg)) + logger.debug("Target: %s, executing pip install %s --ignore-reinstalled --force-reinstall --no-deps" % (target, install_arg)) pip_args += ["--ignore-installed", "--force-reinstall", "--no-deps"] - _call_pip(pip_args, pip_command=pip_command) + + pip_caller.execute(*pip_args) return "ok" -def _call_pip(args, pip_command=None): - if pip_command is None: - import os - python_command = sys.executable - binary_dir = os.path.dirname(python_command) - - pip_command = os.path.join(binary_dir, "pip") - if sys.platform == "win32": - # Windows is a bit special... first of all the file will be called pip.exe, not just pip, and secondly - # for a non-virtualenv install (e.g. global install) the pip binary will not be located in the - # same folder as python.exe, but in a subfolder Scripts, e.g. - # - # C:\Python2.7\ - # |- python.exe - # `- Scripts - # `- pip.exe - - # virtual env? - pip_command = os.path.join(binary_dir, "pip.exe") - - if not os.path.isfile(pip_command): - # nope, let's try the Scripts folder then - scripts_dir = os.path.join(binary_dir, "Scripts") - if os.path.isdir(scripts_dir): - pip_command = os.path.join(scripts_dir, "pip.exe") - - if not os.path.isfile(pip_command) or not os.access(pip_command, os.X_OK): - raise RuntimeError(u"No pip path configured and {pip_command} does not exist or is not executable, can't install".format(**locals())) - - command = [pip_command] + args - - logger.debug(u"Calling: {}".format(" ".join(command))) - - p = sarge.run(" ".join(command), shell=True, async=True, stdout=sarge.Capture(), stderr=sarge.Capture()) - p.wait_events() - - all_stdout = [] - all_stderr = [] - try: - while p.returncode is None: - line = p.stderr.readline(timeout=0.5) - if line: - _log_stderr(line) - all_stderr.append(line) - - line = p.stdout.readline(timeout=0.5) - if line: - _log_stdout(line) - all_stdout.append(line) - - p.commands[0].poll() - - finally: - p.close() - - stderr = p.stderr.text - if stderr: - split_lines = stderr.split("\n") - _log_stderr(*split_lines) - all_stderr += split_lines - - stdout = p.stdout.text - if stdout: - split_lines = stdout.split("\n") - _log_stdout(*split_lines) - all_stdout += split_lines - - return p.returncode, all_stdout, all_stderr +def _log_call(*lines): + _log(lines, prefix=" ") def _log_stdout(*lines): - _log(lines, prefix=">", stream="stdout") + _log(lines, prefix=">") def _log_stderr(*lines): - _log(lines, prefix="!", stream="stderr") + _log(lines, prefix="!") -def _log(lines, prefix=None, stream=None, strip=True): - if strip: - lines = map(lambda x: x.strip(), lines) +def _log(lines, prefix=None): + lines = map(lambda x: x.strip(), lines) for line in lines: console_logger.debug(u"{prefix} {line}".format(**locals())) diff --git a/src/octoprint/server/__init__.py b/src/octoprint/server/__init__.py index 811baaf6..8426628f 100644 --- a/src/octoprint/server/__init__.py +++ b/src/octoprint/server/__init__.py @@ -196,7 +196,8 @@ class Server(): file_manager=fileManager, printer=printer, app_session_manager=appSessionManager, - plugin_lifecycle_manager=pluginLifecycleManager + plugin_lifecycle_manager=pluginLifecycleManager, + data_folder=os.path.join(settings().getBaseFolder("data"), name) ) def settings_plugin_inject_factory(name, implementation): diff --git a/src/octoprint/server/api/__init__.py b/src/octoprint/server/api/__init__.py index d50bb922..a979053a 100644 --- a/src/octoprint/server/api/__init__.py +++ b/src/octoprint/server/api/__init__.py @@ -152,6 +152,8 @@ def performSystemAction(): available_actions = s().get(["system", "actions"]) for availableAction in available_actions: if availableAction["action"] == action: + async = availableAction["async"] if "async" in availableAction else False + ignore = availableAction["ignore"] if "ignore" in availableAction else False logger.info("Performing command: %s" % availableAction["command"]) try: # Note: we put the command in brackets since sarge (up to the most recently released version) has @@ -159,15 +161,18 @@ def performSystemAction(): # workaround again # # See https://bitbucket.org/vinay.sajip/sarge/issue/21/behavior-is-not-like-popen-using-shell - p = sarge.run([availableAction["command"]], stderr=sarge.Capture(), shell=True) - if p.returncode != 0: - returncode = p.returncode - stderr_text = p.stderr.text - logger.warn("Command failed with return code %i: %s" % (returncode, stderr_text)) - return make_response(("Command failed with return code %i: %s" % (returncode, stderr_text), 500, [])) + p = sarge.run([availableAction["command"]], stderr=sarge.Capture(), shell=True, async=async) + if not async: + if not ignore and p.returncode != 0: + returncode = p.returncode + stderr_text = p.stderr.text + logger.warn("Command failed with return code %i: %s" % (returncode, stderr_text)) + return make_response(("Command failed with return code %i: %s" % (returncode, stderr_text), 500, [])) except Exception, e: - logger.warn("Command failed: %s" % e) - return make_response(("Command failed: %s" % e, 500, [])) + if not ignore: + logger.warn("Command failed: %s" % e) + return make_response(("Command failed: %s" % e, 500, [])) + break return NO_CONTENT diff --git a/src/octoprint/static/js/app/viewmodels/navigation.js b/src/octoprint/static/js/app/viewmodels/navigation.js index ee8a4065..c9f3b9b1 100644 --- a/src/octoprint/static/js/app/viewmodels/navigation.js +++ b/src/octoprint/static/js/app/viewmodels/navigation.js @@ -28,15 +28,17 @@ $(function() { new PNotify({title: "Success", text: _.sprintf(gettext("The command \"%(command)s\" executed successfully"), {command: action.name}), type: "success"}); }, error: function(jqXHR, textStatus, errorThrown) { - var error = "

" + _.sprintf(gettext("The command \"%(command)s\" could not be executed."), {command: action.name}) + "

"; - error += pnotifyAdditionalInfo("
" + jqXHR.responseText + "
"); - new PNotify({title: gettext("Error"), text: error, type: "error", hide: false}); + if (!action.hasOwnProperty("ignore") || !action.ignore) { + var error = "

" + _.sprintf(gettext("The command \"%(command)s\" could not be executed."), {command: action.name}) + "

"; + error += pnotifyAdditionalInfo("
" + jqXHR.responseText + "
"); + new PNotify({title: gettext("Error"), text: error, type: "error", hide: false}); + } } }) }; if (action.confirm) { showConfirmationDialog(action.confirm, function (e) { - callback(); + callback(); }); } else { callback(); @@ -49,4 +51,4 @@ $(function() { ["loginStateViewModel", "appearanceViewModel", "settingsViewModel", "userSettingsViewModel"], "#navbar" ]); -}); \ No newline at end of file +}); diff --git a/src/octoprint/util/__init__.py b/src/octoprint/util/__init__.py index a682578a..9b63618e 100644 --- a/src/octoprint/util/__init__.py +++ b/src/octoprint/util/__init__.py @@ -681,4 +681,4 @@ class InvariantContainer(object): return len(self._data) def __iter__(self): - return self._data.__iter__() \ No newline at end of file + return self._data.__iter__() diff --git a/src/octoprint/util/comm.py b/src/octoprint/util/comm.py index 04fda534..14ca4f59 100644 --- a/src/octoprint/util/comm.py +++ b/src/octoprint/util/comm.py @@ -1241,10 +1241,10 @@ class MachineCom(object): self._changeState(self.STATE_DETECT_SERIAL) serial_obj = self._detectPort(True) if serial_obj is None: - self._log("Failed to autodetect serial port") - self._errorValue = 'Failed to autodetect serial port.' + self._errorValue = 'Failed to autodetect serial port, please set it manually.' self._changeState(self.STATE_ERROR) eventManager().fire(Events.ERROR, {"error": self.getErrorString()}) + self._log("Failed to autodetect serial port, please set it manually.") return None port = serial_obj.port @@ -1266,11 +1266,17 @@ class MachineCom(object): for name, factory in serial_factories: try: serial_obj = factory(self, self._port, self._baudrate, settings().getFloat(["serial", "timeout", "connection"])) - except Exception: - self._log("Unexpected error while connecting to serial port: %s %s (hook %s)" % (self._port, get_exception_string(), name)) - self._errorValue = "Failed to open serial port, permissions correct?" + except: + exception_string = get_exception_string() + self._errorValue = "Connection error, see Terminal tab" self._changeState(self.STATE_ERROR) eventManager().fire(Events.ERROR, {"error": self.getErrorString()}) + + self._log("Unexpected error while connecting to serial port: %s %s (hook %s)" % (self._port, exception_string, name)) + + if "failed to set custom baud rate" in exception_string.lower(): + self._log("Your installation does not support custom baudrates (e.g. 250000) for connecting to your printer. This is a problem of the pyserial library that OctoPrint depends on. Please update to a pyserial version that supports your baudrate or switch your printer's firmware to a standard baudrate (e.g. 115200). See https://github.com/foosel/OctoPrint/wiki/OctoPrint-support-for-250000-baud-rate-on-Raspbian") + return False if serial_obj is not None: diff --git a/src/octoprint/util/pip.py b/src/octoprint/util/pip.py new file mode 100644 index 00000000..f670edf7 --- /dev/null +++ b/src/octoprint/util/pip.py @@ -0,0 +1,180 @@ +# coding=utf-8 +from __future__ import absolute_import + +__author__ = "Gina Häußge " +__license__ = 'GNU Affero General Public License http://www.gnu.org/licenses/agpl.html' +__copyright__ = "Copyright (C) 2014 The OctoPrint Project - Released under terms of the AGPLv3 License" + + +import sarge +import sys +import logging + + +class UnknownPip(Exception): + pass + +class PipCaller(object): + def __init__(self, configured=None): + self._logger = logging.getLogger(__name__) + + self._configured = configured + + self._command = None + self._version = None + + self._command, self._version = self._find_pip() + + self.refresh = False + self.on_log_call = lambda *args, **kwargs: None + self.on_log_stdout = lambda *args, **kwargs: None + self.on_log_stderr = lambda *args, **kwargs: None + + def __le__(self, other): + return self.version is not None and self.version <= other + + def __lt__(self, other): + return self.version is not None and self.version < other + + def __ge__(self, other): + return self.version is not None and self.version >= other + + def __gt__(self, other): + return self.version is not None and self.version > other + + @property + def command(self): + return self._command + + @property + def version(self): + return self._version + + @property + def available(self): + return self._command is not None + + def execute(self, *args): + if self.refresh: + self._command, self._version = self._find_pip() + self.refresh = False + + if self._command is None: + raise UnknownPip() + + command = [self._command] + list(args) + + joined_command = " ".join(command) + + self._logger.debug(u"Calling: {}".format(joined_command)) + self.on_log_call(joined_command) + + p = sarge.run(joined_command, shell=True, async=True, stdout=sarge.Capture(), stderr=sarge.Capture()) + p.wait_events() + + all_stdout = [] + all_stderr = [] + try: + while p.returncode is None: + line = p.stderr.readline(timeout=0.5) + if line: + self._log_stderr(line) + all_stderr.append(line) + + line = p.stdout.readline(timeout=0.5) + if line: + self._log_stdout(line) + all_stdout.append(line) + + p.commands[0].poll() + + finally: + p.close() + + stderr = p.stderr.text + if stderr: + split_lines = stderr.split("\n") + self._log_stderr(*split_lines) + all_stderr += split_lines + + stdout = p.stdout.text + if stdout: + split_lines = stdout.split("\n") + self._log_stdout(*split_lines) + all_stdout += split_lines + + return p.returncode, all_stdout, all_stderr + + + def _find_pip(self): + pip_command = self._configured + pip_version = None + + if pip_command is None: + import os + python_command = sys.executable + binary_dir = os.path.dirname(python_command) + + pip_command = os.path.join(binary_dir, "pip") + if sys.platform == "win32": + # Windows is a bit special... first of all the file will be called pip.exe, not just pip, and secondly + # for a non-virtualenv install (e.g. global install) the pip binary will not be located in the + # same folder as python.exe, but in a subfolder Scripts, e.g. + # + # C:\Python2.7\ + # |- python.exe + # `- Scripts + # `- pip.exe + + # virtual env? + pip_command = os.path.join(binary_dir, "pip.exe") + + if not os.path.isfile(pip_command): + # nope, let's try the Scripts folder then + scripts_dir = os.path.join(binary_dir, "Scripts") + if os.path.isdir(scripts_dir): + pip_command = os.path.join(scripts_dir, "pip.exe") + + if not os.path.isfile(pip_command) or not os.access(pip_command, os.X_OK): + pip_command = None + + if pip_command is not None: + self._logger.debug("Found pip at {}, going to figure out its version".format(pip_command)) + command = [pip_command, "--version"] + p = sarge.run(" ".join(command), shell=True, stdout=sarge.Capture(), stderr=sarge.Capture()) + + if p.returncode != 0: + self._logger.warn("Error while trying to run pip --version: {}".format(p.stderr.text)) + pip_command = None + + output = p.stdout.text + # output should look something like this: + # + # pip from () + # + # we'll just split on whitespace and then try to use the second entry + + if not output.startswith("pip"): + self._logger.warn("pip command returned unparseable output, can't determine version: {}".format(output)) + + split_output = map(lambda x: x.strip(), output.split()) + if len(split_output) < 2: + self._logger.warn("pip command returned unparseable output, can't determine version: {}".format(output)) + + version_segment = split_output[1] + + from pkg_resources import parse_version + try: + pip_version = parse_version(version_segment) + except: + self._logger.exception("Error while trying to parse version string from pip command") + else: + self._logger.info("Found pip at {}, version is {}".format(pip_command, version_segment)) + + return pip_command, pip_version + + def _log_stdout(self, *lines): + self.on_log_stdout(*lines) + + def _log_stderr(self, *lines): + self.on_log_stderr(*lines)