diff --git a/ChangeLog b/ChangeLog index 0a8e1386..cb75142e 100644 --- a/ChangeLog +++ b/ChangeLog @@ -1,1105 +1,1106 @@ --- Version 0.7.2 - UNRELEASED +- Avoid some logged warnings from the R backend on exit - Add "Check installation" wizard to test for several common basic instation issues all in one place - Add rkward internal package location to end of library search path, to make it accessible to help.search() - Add menu action to open any supported file type, directly - Support using QWebEngine instead of QtWebKit (still supported) - elements in plugins may now also contain clickable links, including rkward://-scheme links - The new code hinting features from version 0.7.1 are now also available in the console * TODO: I'm not quite happy with the behavior e.g. when typing "?print", yet (requires twice) - On unix-systems, RKWard can now be run without installation - Kate addons are now supported within RKWard. Intially, search-in-files, snippets, and projects are loaded by default * TODO: Some plugins still have UI issues. E.g.: - SQL plugin fails to add toplevel menu - Text filter plugin always shows menu entry, but should be limited to script windows * TODO: Instruct packagers that they should add a dependency on kate, or at least kate plugins - Fixed: Avoid shorcut clash with kate part by removing conflicting (default) shortcuts, automatically --- Version 0.7.1b - May-25-2020 - Workaround to avoid bug in R 4.0.0 when using for loop on top level context - Fixed: Hang when opening an empty workplace - Fixed: Crash when mousing over code preview window with focus-follows-mouse enabled --- Version 0.7.1 - Jan-23-2020 - Code hinting in script editor windows has been reworked, and now also completes argument names - Instead of installing rkward R packages at build time, install them at runtime, when needed - Add new commandline option "--r-executable=auto" for auto-detection of R installation - Fixed: Underscore (_) was not accepted in names of newly created data.frames - element in save file mode gains checkbox to control overwriting of existing files - Fixed: element could not be set to not required - Fixed: max.print option was not saved, correctly, when set from RKWard settings - Add command-line option --autoreuse to avoid shipping two separate .desktop files - - New plugin element for selecting arbitrary string values (otherwise almost identical to ) - can be set to accept the same object several times. Used in scatterplot plugin. - New R function rk.embed.device() for manually embedding graphics devices in RKWard - Fixed: R backend would exit immediately, without meaningful error message, if there is an error in .Rprofile (or Rprofile.site) - Fixed: Installing suggested packages from the package installation dialog was broken - Fixed: Selecting a mirror via the "R packages" settings page would not work when prompted for package installation form the backend - Remove support for compiling RKWard in a single process (threaded) variant. This was not used / tested since RKWard 0.5.5 - Shortcuts for the "Run ..." actions have been changed for better cross-platform compatibility - The script editor's "Run line" and "Run selection" actions have been merged - Add UI for configuring default graphics device, and embedding of standard graphics devices. - New RKWard native on-screen device (RK()). This is the default on-screen device in RKWard, now. - New R function rk.without.plot.history() for turning off plot history, temporarily - Add command line option --backend-debugger --- Version 0.6.1 - Apr-02-2013 - Add option to force-close a graphics window - Add plugin for subsetting data.frames by rows or columns - On the Windows platform, add an new (experimental) binary startup wrapper (rkward.exe) - Revert to building R packages form source on Mac OS X by default (controllable via compile-time option) - Fixed: lattice plots would not be added to the plot history, correctly, for some versions of lattice - Fix crash when trying to print, and neither okular, nor kpdf are available - Added support for loaded namespaces that are not attached to a loaded package - Pluginmaps can specify their "priority". Pluginmaps with low priority will not be added automatically, when found. - Pluginmaps can other pluginmaps based on their id (for cross-package inclusion) - Added new element for dynamic version checks within a plugin (R and RKWard versions, only, so far) - Add guard against accidental usage of the standard distributed pluginmaps in a later version of RKWard (installed in parallel) - Easier (de-)activation of RKWard plugin maps using checkboxes (Settings->Configure RKWard->Plugins) - Broken or quirky .pluginmap files are reported to the user, completely broken maps are disabled, automatically - Implement basic dependency handling for plugins / pluginmaps - Added support for the upcoming R 3.0.0 - Added logic element to switch between several target properties (or fixed values) based on the value of a condition property - Sort plugin gains option to sort data.frames by more than one column at a time, and options for type conversion - Add in-application debug message viewer (targetted at (plugin) developers) - Add setting to customize initial working directory - Windows only: Add UI-checkbox for R's "internet2"-option - New functions getString(), getList() and getBoolean() for fetching data in plugin scripts - Boolean properties now return a numeric, not labelled representation of their value, by default. es should be unaffected. - Added GUI element for entering a set of options for an arbitrary number of items - Reduce CPU usage of pluings while idle - Fix conversion from Numeric to Factor in the data editor - In the data.frame editor, columns containing invalid values are now highlighted in red - Fixed: If none of the previous plugin maps could be found on startup, re-add the default - Added GUI element for entering matrix or vector data in plugins - Improve key handling while editing factor levels in a data.frame - Added utiltity function rk.flush.output() - RKWard is now categorized as Science/Math/Numerical Analysis in its .desktop file - Fixed: Yet another fix for hard-to-read function argument hints - Fixed: Device history was not working with more recent versions of ggplot2 - Fixed: Option to include suggested packages in install was mis-labelled "install dependencies" - rk.set.output.html.file() gains argument to allow addition of custom content to the html header --- Version 0.6.0 - Oct-24-2012 - Fixed: Entering "0" as propabilities (quantiles) vector in distribution plugins would cause error message - Preview device windows will display some status information (most importantly, warnings or errors) - Most plot plugins gain options to control margins and tick label orientation - Added option for installing packages from source (implicitly enabled on Unixoid platforms) - Fixed: Wrong handling of carriage returns ('\r') in the console window - Fixed: Spinboxes had wrong initial values - Omit comments on missing function calls in dialog code windows (e.g., if prepare() is unused, there's no "## Prepare" in the output either) - Output markup is now more XHTML compliant and easier to parse - Also save cursor position, folding, etc. for scripts. Note: Implementation details may be subject to change. - New function rk.list.labels() to retrieve column labels in a data.frame - rk.get.label() will now return an empty string ("") instead of NULL, in case no label is assigned - Fixed: Changed configuration settings would not be saved for script editor - Fixed: One character missing in R commands on lines longer than 4096 characters - Fixed: "Next"-button in wizards would remain enabled while settings are missing on a page - Fixed: Dynamic pages in a wizard would cause a layout bug on the first page - Do not offer to restore individual removed columns of a data.frame opened for editing - Fixed: Plot history and graphical menus broken in some cases with R 2.15.0 - Fixed: If the rkward package was loaded in a plain R session, q() and quit() still work - combined all Wilcoxon/Mann-Whitney-tests in one plugin (from previously two separate plugins) - Added polyserial/polychoric correlations to correlation matrix plugin - Added more compression options to the "Save objects as R code" plugin - Added MacPorts support, see README.MacPorts and bundle build script in the macports folder - Added dynamically generated table-of-contents menu to output document - Fixed: Would not show output of system() commands to stderr on the console (on Unix-like systems) - Allow some markup inside elements in plugins, and auto-add breaks only for duplicate newlines. - Reorganized t-test plugin, and add support for single sample t-tests - Box plot plugin gains more options for adding means - Fixed: Function argument hints for the second half of the parameter list would not be quoted, correctly - Improve keypress handling issues in the R Console, when the cursor or a selection is outside the editable range - Fixed: Failure to open workspaces with non-latin1 filenames from the command line - Fixed: Saving / restoring workplace layout would fail when saving to directories with unusual characters - Only install translations which are at least 80% complete (not counting strings in plugins, which are not yet translatable) - When asking for workspace file to open, use appropriate file filter - When configured to ask for workspace to open on startup, don't prompt to save, first - Fixed: potential crash when clicking "Select all updates" in the package installation dialog - Fixed: potential crash in object name completion under certain conditions - Simplified the "Sort Data" plugin, and added a help page - Fixed: On Windows, detached windows would sometimes be positioned with the menubar outside the upper screen edge - Added GUI support for inspecting the call stack during debugging - The backend executable is no longer linked against KDE libraries - Objects, which are not acceptable in a varslot, will still be shown, there, with a warning --- Version 0.5.7 - Oct-23-2011 - Fixed: Would not system-defined text color for argument hints - Do not treat arrays (which are not a matrix) as hierarchical named objects in the object browser - Do not analyse more than 100000 name child-objects per object (avoids hangs on such extreme data) - Fixed: Problems with using mclapply() inside RKWard - Offer to add a new library location, automatically, if location selected for installation is not writable - Reworked package installation / update dialog - Fixed: Integrated help browser would not update navigation history when following page internal links - Documentation on writing RKWard plugins is now accessible, locally - The file filter for R script files is now configurable, and includes *.Rhistory, by default - More obvious coloring of variable entry fields requiring user input in plugins - Default size of the code display in plugin dialogs has been increased - Removed support for downloading plugins using GHNS - Added support for RKWard plugins shipped inside R packages - Fixed: PDFs and many other types of documents linked from help pages would not be opened, correctly - Added support R 2.14.x - Removed option to set options("printcmd") - New option to run arbitrary (setup) commands in each session - Added new pseudo graphics device "rk.printer.device" to provide printing via the KDE printer dialog - Fixed: Printing was broken when kprinter is not available - Support the results list new help.search() in R 2.14.x, which includes vignettes and demos - rk.edit.files() and rk.show.files() gain parameter "prompt", which can be used to suppress user prompt - Added function rk.demo(), which is similar to demo(), but opens the example script in a script editor window - Fixed: Some plugin dialogs would not become visible, or be shown in very small size, with some window managers - Fixed: Potential crash when using context menus with "focus follows mouse" window activation policy - Added shortcut Ctrl+Enter to insert a line break and submit from a script editor - Reduce CPU usage while idle - Bar plot, dot chart, pie chart, and Pareto plot plugins gain more tabulation options - rk.results() now prints rownames, where appropriate - "frame" elements in plugins can now be made checkable - Disabling or hiding a plugin component makes it non-required, implicitly - Box plot gains option to dogde grouped boxes - Simplify internal handling of system() and system2() output - Simplify code produced by several plugins - Fixed: Occasional duplication of first letter of keyword, when using "Copy lines to output" from the console window --- Version 0.5.6 - May-30-2011 - Fixed: Would crash when searching for a term with quotes in the help search window - Add simple man page for command line options - Fixed: Potential crash when creating plots from a tcl/tk based GUI - Fix compilation on FreeBSD - Replace the R console's "Copy selection cleaned" action with "Copy commands, only", which will omit any non-command regions in the copy - Add "Copy lines to output" action to copy highlighted lines from the R console or script editors to the output window - R commands and their output can be "carbon copied" to the output window - On Windows, RKWard will detect, and offer to disable "native" file system dialogs - Object browsers that show only the global environment now hide the ".GlobalEnv" item - Fixed: Function argument hints would not be shown in some corner cases - Added function rk.print.code() to write highlighted R code to the output window - Box plot plugin gains support for grouped outcome data - Fixed: Pressing Ctrl+C would not reset syntactically incomplete commands in the R console - Crosstabs N to 1 plugin gains options to compute proportions and margins (thanks to Andrés Necochea) - Added convenience R function rk.list() to allow simplification of plugin code - Added stack-based window switching using Ctrl+(Shift)+Tab; this replaces the old "Next Window" and "Previous Window" actions - Fixed: Graphics device windows would disappear when trying to attach them to the main window with some versions of Qt - Fixed: tcl/tk widgets would lock up after running commands in the R Console - Provide a template for bug reports, containing standardized information on the RKWard setup in Help->Report Bug... - When loading a (local) workspace, change the working directory to the directory of the workspace (configurable) - Include the sidebar position of tool windows when saving / restoring the workplace layout - "Pending jobs" tool window is not longer shown in the bottom sidebar by default - Support removing individual tool windows from the sidebars, completely - Add an action to interrupt all (current and pending) commands - Fixed: When saving the workplace while closing the session, the workplace layout would not always be saved correctly - File browser context menu gains many more KDE standard actions (KDE 4.3 and above, only) - Auxiliary binaries rkward.bin and rkward.rbackend are no longer installed into /usr[/local]/bin/ on Unix - Fixed: Object name completion would abort early in the script editor with KDE 4.4 - Support function argument hinting for R primitives - Support package namespaces in object name completion and workspace browser - Support S4 slots in object name completion and workspace browser - More correct handling of quotes in object name completion - Support plot history for ggplot2 plots - Be less pro-active about fetching structure information on R objects in the workspace - Allow to browse arbitrarily deeply nested environments in the object browser --- Version 0.5.5 - Mar-15-2011 - Fixed: Crash when several environments on the search path have the same name - Re-organize the default toolbar actions of the main window - Detached windows are now restored in detached state when restoring the workplace layout - Workplace layout is now saved in a separate file, instead of in a hidden object inside the .RData file - Added R functions rk.save.workplace() and rk.restore.workplace() to save / restore a set of document windows - RKWard now tries to detect, when a workspace has been moved to a new directory, and adjust the paths restored script editor windows, accordingly - Fixed: Potential crashes when changing length of a data.frame that is currently opened for editing from R code - All pages in the package installation dialog now support sorting and keyboard search - Fixed: Converting from factor to string in the data editor set values to numeric, internally - Allow entering factor labels instead of only numbers, when editing factors in the data editor - Display logical values as "TRUE" and "FALSE" in data editor, and accept "T"/"TRUE"/"F"/"FALSE" in addition to "0"/"1" while editing - Add support for R functions loadhistory(), savehistory(), history(), and timestamp () - Do not load .RData file from current directory by default (configurable) - Fixed: On Windows, a wrong locale for LC_NUMERIC would be applied, resulting in malfunction of pdf() and postscript() - Fixed: Did not use system configured background color in data editor - Fixed: New columns would be always be added to the right, instead of to the left, in the data editor - Fixed: Would crash when trying to edit 0 row data.frame - Fixed: Output generated by external processes (system()) was not shown on the console - Fixed: Converting a variable to factor in the editor would drop existing levels, silently, and lead to NAs - Fixed: Cursor would keep jumping to the end, when typing a filename in the filename selection field in plugins - Fixed: Submit button would not become enabled in "Basic Statistics" plugin - Moved automated testing framework moved to a dedicated R package "rkwardtests", and added documentation - Support pasting to the middle of the current command line in the R Console - Better handling of script-editor commands while the R Console is busy - Correct interleaving of multi-line commands and the corresponding output in the R Console - Fixed: Portions of multi-line commands entered on the R Console would be run twice under some circumstances - Fixed: Lockups or crashes when using packages RGtk2 or rJava on the X11 platform - Fixed: R task callback handlers were not called - GUI frontend and R backend now run in separate processes - Re-write of large portions of the R embedding code - Fixed: Did not respect system settings for tooltip background color in some styles - Fixed: Installation would continue, seemingly successfully, even if R support libraries could not be installed - Fixed: Function argument hints would sometimes persist after closing a script window - Fixed: Would fail to analyse structure of ReferenceClass-objects - Fixed: "Vector" mode in "Paste special" action did not work correctly - Attempt to save workspace on crashes - Also try to relay SIGABRT and SIGILL to the proper signal handlers - Removed the remainder of the PHP scripting backend - Fixed: Would crash on tcl/tk events in some cases --- Version 0.5.4 - Oct-04-2010 - Fixed: Running user commands would crash with R 2.12.0 - Added full-featured plot history for the onscreen graphics device - Fixed: Potential crash when editing data-frame with duplicate column names - Added option to specify default onscreen graphics dimensions - Fixed: Graphics windows used to assume a minimum size of 640*480 pixels, even if a smaller size was specified - Added option to disable function argument hinting - New functions rk.show.message() and rk.show.question() for user interaction from R code - New options for scripting GUI logic in plugins - The current object of an active data editor can be referenced in plugins - Fixed: Placement of several menu items was broken - again - with KDE 4.4 and above - Allow sorting of results in help search window - Fixed: CPU usage would go to 100% for no good reason under certain circumstances - The save-object selector in plugins now allows to save as part of a data.frame / list - Fixed: On some systems, dev.off() would spawn two new graphics windows - Support row names in the data.frame-editor - Fixed: When starting with an empty table, RKWard would sometimes claim that this object has been removed - Fixed: Would crash when trying to configure toolbars under certain circumstances (workaround for bug in kdelibs) - Fixed: Crash while analysing some objects returned by XML::xmlParseTree() for invalid XML - Fixed: Error while installing packages with R 2.11.0, when archiving packages in a non-existing directory - Added option to autosave script files (enabled by default) - The tabbar in the main window now shows a context menu with options to close/detach a window - The tabs in the main window can now be re-ordered by dragging with the mouse (left click if compiled with Qt 4.5 or above, middle click for earlier versions) - Added alternating row backgrounds in data.frame-editor --- Version 0.5.3 - Apr-30-2010 - Fixed: Windows->Activate->Window left / right actions were always disabled - File browser window saves settings on exit - Several windows-specific bugs were fixed - Plugin dialogs are shown with a larger initial size - Add close buttons to each tab in the main document view area (not available in KDE 4.0) - Make "print", "export as HTML", "dynamic word wrap", and "increase/descress font size" available for the console window - Use a native menu, instead of the default TclTk-menu - Fixed: Newly created variables were not properly updated when closing and re-opening the editor - Added "paste special" action to script editor and console for pasting R vectors and matrices from spreadsheets - Fix order of menus for detached windows - File->Open R Script File now allows to specify the character encoding to use - Fixed: No entries were added the recent script/workspaces actions after "save as" - Initialize the output file with an appropriate encoding specification - Add SVG support to export (graphics) plugin - Add basic settings format settings for graphics output - Convert all plugins to use ECMAscript instead of PHP; RKWard no longer depends on PHP - Fixed: Frequent crashes while running automated plugintests - Fixed: Filenames without extension would not be shown in file dialogs - Fixed: Calling "fix(my.fun)" would remove comments - Fixed: Removing "@CRAN@" from the repositories would break package installation --- Version 0.5.2 - Oct-26-2009 - Add Stata data file import plugin (by Michael Ash) - All directly accessible plugins now have at least one automated test - Add support for the dynamic help server introduced in R 2.10.0 - Assorted minor fixes and improvements to several plugins - Fix deadlock while handling some Tcl events - Plugin dialogs close automatically after submitting (by default) - Fetching object structure is much faster for very large data.frames - Fix crash when loading certain packages on Windows - Fix some potential path issues on Windows - "Analysis" menu was restructured, slightly - Debug output (previously sent to stderr) is now written to a temporary file - On plugin help pages, display a link to invoke the plugin - Tolerate missing libraries in testing framework - Double-clicking an item in the workspace browser now opens an object viewer, or (if possible) editor - Safeguard against removal of essential packages via the GUI - Add context menu action to unload packages in the workspace browser window - Add shortcut to the load / unload packages dialog also in the workspace menu and the workspace browser context menu - Make toolbar buttons configurable - Fixed: Console window would sometimes remain in partially active state after piping commands --- Version 0.5.1 - Aug-04-2009 - Support compilation on Windows - Add framework for automated plugin testing, and some first plugin tests - New command line option "--evaluate " mostly for the purpose of automated testing - Remove support for R 2.6.x and earlier - Add basic checks for a correct installation of the RKWard resource files - Remove "What to expect" dialog at startup - Make keyboard shorcuts configurable - Add option to add only single line commands from script editor to the console history - Add action to change working directory to that of the current script file - Fixed: No warning was shown, when an open script file was changed on disk, externally - Fixed: Opening most file types (e.g. PDF) from the help browser was broken - Fixed: Make built-in editor work again for file.edit () - More plugins now write a header to the output window - Add convenience function "makeHeaderCode()" for use inside plugins - Adjust some icons - Plugins can be invoked from R code (mostly for internal purposes) - Add "Run again" link for plugin generated output - Fixed: All objects in .Globalenv would be revisited if a single object was added / removed - Fixed: Screen device in rkward was not seen as interactive by R --- Version 0.5.0d - May-10-2009 - Fixed: Container of detached windows would sometimes remain after reattaching - Fixed: Global toolbar style was not honored - Fixed: Auto-scroll missing in progress dialogs and data editor - Fix jumping toolbar - Fixed: Several harmless warnings were silenced - Fixed: Argument hints would always stay on top regardless of focus - Fix crash when reaching bottom of command history in context sensitive search - Fix crash while inserting rows in a data.frame with string variables --- Version 0.5.0c - Mar-30-2009 - Fix assorted installation problems - Updated translations: German, Spanish - Fix immediate crash issue with R 2.9.0 (requires recompilation against R 2.9) - Fixed some shortcut problems with KDElibs 4.2.x - Fixed: Several icons would not be loaded - Fix device window capturing Qt 4.4.3 (possibly other versions were broken as well) - Fix device window capturing with R 2.8.0 - Extended boxplot plugin - Support data.frames in barplot plugin - New set of plugins: Item Response Theory (by Meik Michalke) --- Version 0.5.0b - Apr-20-2008 - Support compilation with R 2.7 - New plugin: Hodrick-Prescott Filter - Fix a problem with plugin help pages not loading correctly - Better resize handling in progress and R input (readline()) dialogs - Somewhat better handling of the R event loop (esp. when calling "require(not_yet_installed_package)") --- Version 0.5.0a - Jan-23-2008 - Fixed: no objects were selectable in most varslots - Compilation: check for minimum kdelibs version during cmake stage --- Version 0.5.0 - Jan-21-2008 - add support for a focus-follows-mouse activation behavior of the MDI windows (Settings->General) - ability to mark arbitrary "blocks" in script editor and run them with a shortcut - make sure that R only handles SIGSEV, when the R thread is active. Should give better backtraces and less hangs - object viewer was redesigned to use space better, and only fetch the print-representation when needed - more icons added - data.frame editor can be set to read-only mode - code completion in script editor shows icons for the different object types - settings dialog is now shown in "page tree" mode - port to KDE 4 ---- KDE 3 versions --- Version 0.4.9c - Mar-18-2010 - Backport all plugin changes and additions up to RKWard 0.5.2 - Add support for the dynamic help server introduced in R 2.10.0 - Backport some improvements in event handling - Fix deadlock while handling some Tcl events - Fix: Warn before saving file that was modified on disk, externally - Fix: Do not try to open mimetypes other than text/html directly in the help browser. - Fixed: Screen device in rkward was not seen as interactive by R --- Version 0.4.9b - May-10-2009 - Workaround: Deactivate all filemanagement shortcuts in the File Browser. These would sometimes trigger when they should not. - Fix crash while inserting rows in a data.frame with string variables - Fix immediate crash issue with R 2.9.0 (requires recompilation against R 2.9) - Fix device window capturing with R 2.8.0 - Extended boxplot plugin - Support data.frames in barplot plugin - New set of plugins: Item Response Theory (by Meik Michalke) --- Version 0.4.9a - Apr-20-2008 - Support from compilation with R 2.7 - New plugin: Hodrick-Prescott Filter - Fixed some lockups during startup --- Version 0.4.9 - Jan-14-2008 - Improved user interface for CSV import plugin - Support drawing grids in histogram and stripchart plots - New plugin: Levene test - New plugin: Jarque-Bera test (by Germán Márquez Mejía) - Fixed: Crosstabs plugin produces better output - Fixed: use.value.labels option was wrong in SPSS import plugin - Fixed: SPSS tables would only be imported to the workspace, when the edit option was checked - Fixed: correct installation of rkward.desktop file - Use the KDE default fixed spacing font for command log and object viewer --- Version 0.4.8a - Nov-11-2007 - More reliable C stack limit detection on some systems - Fixed: Console accepted pasted input while a command is running - Fixed: Analysis->Moments->Moment plugin did not use the order-parameter - Fixed: Crash on simple assignments in globalenv() with R < 2.4.0 - Fixed: Meta information (labels) did not get shown correctly in the data editor - Add command line option "--debugger" for debugging purposes - Fixed: Pressing the add button in a varslot with no object selected could crash RKWard --- Version 0.4.8 - Oct-03-2007 - New plugins: pie chart, dot chart, crosstabulation, two time-series tests, and basic linear regression - barplot plugin allow specification of custom labels - commands run through the console from the script editor will now be added to the command history (this can be turned off in the settings) - fixed: promise objects (created by delayedAssign()) directly in globalenv() would be forced early - make wrapper script work, when konsole is not installed - correct syntax highlighting for ::: and NA_integer_ and others - silence some GCC 4.2 warnings - fixed: crash when editing a data.frame with logicals (logicals are still mishandled, but less severe) - fixed: would not work with R 2.6 - new translation: Simplified Chinese (contributed by Roy Qu) - more compilation fixes for GCC 4.3 - object viewer also shows summary information, gains an update button, and MDI integration - CLT plugins allow drawing grid - add max.print option to R-Backend settings dialog - syntax highlighting is minimally faster for long outputs - graphics export plugin gains option for EPS compatibility - fixed: inserting / removing rows in the data editor in the presence of invalid fields in later rows would lead to a crash - fix some internal problems with enabledness/disabledness of plugin components - some complex plotting plugins now start up noticeably faster - fixed: sometimes detecting created X11() windows would fail, esp. under high load - create the workspace browser only when it is first shown - support for including files and snippets in xml files - all file selection line edits gain filename-completion - add a basic file selector window - show the focus indication (thin red border) also for detached windows - tool windows can be closed and detached in the same way as regular windows - fix a crash with detached tool windows - add command line option --disable-stack-check for systems where R C stack checking is buggy - the modifier "not" for boolean properties returns a (negated) boolean sub-propery instead of just a string - add possibility to blacklist packages from fetching structure information. Package GO is blacklisted by default. - when fetching the structure of "promise" R objects, don't keep them in memory - optimize the function to retrieve the structure of R objects (for display in the object browser, and completion) - Messages, warnings, and errors for plugin commands are shown in the output, instead of in a dialog --- Version 0.4.7a - May-07-2007 - add sanity check for the detected stack limits - fixed: one some systems rkward would crash or not compile due to missing fortran symbols - fixed: starting the scatterplot plugin in wizard mode would crash rkward - tab-completion in the console will do partial completion, if all potential completions have a common start - fixed: file-name completion would always assume the home directory as the current directory - update turkish translation - compilation fix for FreeBSD (thanks to Thierry Thomas) - fixed: when executing commands line by line from the script editor, line breaks would be omitted --- Version 0.4.7 - Apr-11-2007 - some fixes for GCC 4.3 - fixed: the presence of user objects called "missing", "assign", or "get" would confuse some RKWard internals - several existing plugins have been improved in various ways - use maximum permissible width in the object viewer - fix a crash that occurred during startup under some circumstances - when flushing the output, also delete all graphs - add support for a "technical details" section in plugin help files - add option to enable / disable single options of a radio or dropdown control in plugins - t-test plugin now also allows to calculate a paired test - show current working directory in the statusbar - add run selection action to all HTML windows - add custom icons for run line/selection/all - make sure not to run parts of an incomplete user command. It would lead to all sorts of subtle problems - if an incomplete command was piped through the console, the console does not become blocked for piping - make "run current line" advance the cursor to next line after running - warnings and errors are highlighted in command log / console (warnings only for R >= 2.5.x) - plugin generated commands are run in a local() environment, allowing for more concise code - also support filename completion in the console (mode of completion is determined by heuristic means) - initial support for R's Sys.setlocale(), and encoding of CHARSXPs - in help files, a short title for sections (for the navigation bar) can be defined - the currently active window is now highlighted using a thin red border - fixed: changing the type of a column in the data editor was broken - fixed: quitting while a plugin was still active would sometimes crash rkward - new shortcuts for window navigation: next/previous window, activate console, activate command log, etc. - new functions rk.describe.alternative () and rk.print.literal () for use in plugins - new plugin: Phillips-Perron test - new plugin: Mood test - new plugins: CLT plots for most distributions - new plugins: variance tests (F test (var.test), Bartlett test, Fligner test) - new plugin: generic plot - new plugin: Add grid to plot - new plugin: pareto chart - new plugin: stem-and-leaf plot - make readline dialog store last size and position - new plugins: outlier tests (chisq.out.test, dixon.test, grubbs.test, outlier) - support R's mechanism for checking the C stack limits (only with R 2.3.x or greater, and not on all platforms) - new plugins: support of all tests from moments package - several new help pages - better error handling in PHP backend - fixed: depending on some buffer sizes, plugins could get stuck while generating code - fixed: plugin browser type="dir" would not work correctly - import SPSS and import CSV plugins gain option to open object for editing, automatically (checked by default) - warn when opening very large objects (with more than 250000 fields; this limit is configurable) - add a "copy" tag to facilitate writing plugins with both dialog and wizard interfaces - fixed: graph previews would stop working when the interface is switched from dialog to wizard or vice versa - new options for plugin dialogs: Configure whether code display is shown by default, and at what size --- Version 0.4.6 - Feb-15-2007 - make sure any LC_ALL environment setting will not be applied by SCIM plugin (if installed on the system) - fix crash after changing storage type of an object in the data editor - single line input fields in plugins no longer accept new-lines - support for calculating p-values in correlation matrix plugin - fixed: pressing F8 in the console would remove input focus from the console - pressing tab changes focus while in a single line text input field - fix several small problems with detached windows - fix a problem with keeping the list of objects up to date - improvements to spinbox: step size is adjusted dynamically, and no arbitrary limits in real mode - fixed problem with specifying y axis limits in several plugins - simplified CSV import plugin - new plugin: import SPSS files - new documentation pages: console - various improvements to existing distribution and distribution plot plugins - new plugins: distribution plots: beta, binomial, cauchy, exponential, gamma, geometric, hypergeometric, logistics, lognormal, negative binomail, tukey, uniform, weibull, wilcoxon - fixed: installing packages as root would not work, when $R_HOME is not defined - new plugin element to select name of an R object to save to - new plugin: Ansari-Bradley two-sample test - place internal objects .rk.rkreply and .rk.available.packages.cache into rkward package environment - support for preview functionality in graphing plugins - new plugin: barplot - new plugin: export contents of an X11 device to several different file formats - plugins can now be context sensitive (e.g. work on a particular x11 device) - new plugin: Wilcoxon Rank Sum test and Wilcoxon Exact Rank Sum test - new plugin: scatterplot matrix - new plugin: correlation matrix plot - safer destruction of R backend on quitting - fix compilation for the upcoming R 2.5.0 - show full error message for syntax errors (only if compiled with R 2.4.0 or newer) - more correct auto-printing of values for user commands (only if compiled with R 2.4.0 or newer) - more correct handling of user commands consisting of several statements (only if compiled with R 2.4.0 or newer) - add ability to search command history context sensitively - add possibility to use dropdown lists in plugins --- Version 0.4.5 - Jan-21-2007 - change shortcuts for Open/Save Workspace to Ctrl+Shift+O/S to avoid clash with Open/Save Script File - remove "Configure Packages" option from Workspace menu, as it is also in Settings menu - fixed: non-local workspaces would not be loaded - fixed: specifying a workspace to load on the command line would only work for full urls - new Console RMB actions: "copy selection cleaned" and "run selection" - added a basic help system, and some introductory help pages - added plugin for skewness and kurtosis - added several plugins for distribution analysis - added plugins for stripchart & density plot - enhancements to existing plot plugins - added refresh output and flush output as toolbar actions - removed the "output" menu; entries were moved to "Edit" and "View" - always show all top level menus - code generated for most plugins cleaned and simplified - new function rk.results () for use in plugins - fix: the output window would not refresh changed plot images (unless closed and reopened) - fix several minor issues as identified by the "krazy" source code checker - allow varslots to request objects of specified dimensionality and length - fixed: integer spinboxes with starting values greater than 1 would get unreasonable change steps - fix compilation with some versions of KDE / GCC - small improvements to syntax highlighting definition - in console, make cursor go to empty next line after pressing enter, before there is output (more like in the real console) - add "run selection" option to command log - script editor commands are piped through the console (by default; this is configurable) - assorted cleanups --- Version 0.4.2 - Dec-04-2006 - code generated by scatterplot plugin is simplified somewhat - add comment headers to the sections of commands produced by plugins - if a user command results in the output html file to be modified, auto-refresh output - add RMB menu to script editor windows - add options "clear" and "configure" in the console RMB menu, and the command log RMB menu - add option to save workplace layout not per workspace, but at the end of the session - add options ("printcmd") to settings - fix help menu for detached windows - R X11 device windows are now managed by rkward, with the following initial features: - R X11 device windows: attach to / detach from workplace (they start detached) - R X11 device windows: toggle between normal resize mode and (settable) fixed size with scrollbars - R X11 device windows: menu options to copy / print / duplicate / save / activate devices - fixed: selecting Window->Close in a detached window would only close the internal view, not the window - when detaching a window, give it a more reasonable size - do not open the same script url twice (instead the corresponding window is raised on the second attempt) - fixed: when closing a detached script editor window, you would not be asked to save changes (if any) - fixed: occasional crash when re-attaching a script editor window - new translations: Greek, Catalan - updated translations: German, French - added basic plot facility for ECDF (Empirical Cumulative Distribution Function) (menu plots) - added ptukey and qtukey (Studentized Range Distribution (Tukey)) (menu distributions) - spelling corrections in distributions menu - reorganized "Descriptive Statistics" plugin to use tabs for better usability - added constant for mad in descriptive statistics plugin - fix bug in rk.get.label () - in script editor, do not provide symbol name completion while in comments - properly support active bindings in globalenv () - if not started in a tty, spawn a konsole. R will go into non-interactive mode otherwise --- Version 0.4.1 - Nov-05-2006 - allow specification of title/subtitle in plot options plugin (currently used by boxplot, and histogram) - fix bug in beta probabilities plugin - fix (some?) compilation problems with KDE 3.2 - when packages have been newly installed, show them in the load / unload packages tab immediately - fix to plugin input lines sizing, and visibility of scrollbars - on startup, show window maximized if no size stored (i.e. very first startup) - prevent infinite recursion when parsing structure of environments - fix for compilation on 64bit architectures - remove misleading "About KDE" dialog, and replace report Bug dialog with correct information - don't crash when closing a detached window - function argument hinting in the console and script editor - as you type completion of R symbol names in the script editor - tab completion of R symbol names in the console - added R function rk.edit (x) to open object x for editing in rkward - fixed: an empty table created before the object list was updated (esp. at startup) would be thought to have been removed in the workspace - object data changed in the console (or by other means) is updated to the editor automatically, if the object is opened for editing - fixed: back / forward buttons would sometimes remain after closing a help window - fix some focus problems - provide close buttons in tab bar --- Version 0.4.0 - Oct-17-2006 - many new distribution plugins - all distribution plugins now use a free-text field, allowing you to enter a vector of probabilties / quantiles - another fix to cancelling during readline () calls - invalid values are stored in a separate attribute instead of changing the storage mode back and forth - storage mode for RKWard meta data was changed - RMB option to search help on objects from packages in object browser - show tooltip information on objects in object browser - do not crash on call to "browser ()" - the object browser is updated automatically when needed - allow some more configuration in object browser - include all package environments in object list - use more efficient data passing to synchronize object list - add (ugly) items for functions and lists to object browser - internal cleanups (partially to make porting to KDE4/Qt4 easier) - when saving/loading the workspace, save / restore all windows (not just data editors) - remember workspace url after save as - fix a long standing (but hard to trigger) crash while closing data editor window - fix "Open R Script File" filename ending filter - add mismatching brace detection to syntax highlighting - yet another crash on more than 100 rows bug fixed - RKWard should now be fully UTF-8 aware. All characters are passed to and from R reliably --- Version 0.3.7 - Sep-17-2006 - added Turkish translation - avoid lockups when cancelling a readline () - show output context (if any) when the backend asks for input via readline () - link against libRblas.so if it is found in the R libs dir (needed for some compiles of R 2.4.0) - remove whitespace from start and end of plugin generated commands and do not evaluate empty commands - do not raise the command log more than once for a given command producing continuous output - small improvements to command history in console - deal better with commands producing large amounts of output - small fixes in several plugins - package installation/update: semi-automate package installation in response to "require ()" calls - package installation/update: the online list of available packages is cached up to one hour (and only as long as the relevant options are unchanged) - package installation/update: give better progress indication - package installation/update: package dependencies are handled correctly when installating new packages - package installation/update: packages can be installed to custom library locations - fix spurious line breaks in output in command log and console - Improved syntax highlighting defintion including code folding - fix compilation with GCC 4.1 - don't crash on more than 100 rows of data in the editor (off by 2 bug in storage allocation) - correctly initialize the internal NaN representation (would sometimes be initialized to 0, resulting in 0s not to be shown in the editor) - engine status bar label ("R engine busy/idle") now changes background color according to state - honor DESTDIR setting while installing rkward R package, and add configure option --with-r-libdir - only link against -lRlapack, if that is available - commands producing lots of output would sometimes be impossible to interrupt. Fixed - show command output immediately in command log - remember location where R script/workspace files were last opened, and start file-selection dialogs there - sometimes the last level in the factor levels editor got lost. Fixed --- Version 0.3.6 - Apr-23-2006 - several fixes to the distributions plugins - remove debian dir from source distribution again - remove one level of menu nesting for save/load R Objects plugins - check for and handle some strange R parse errors that could crash rkward - make Ctrl+C copy work in the code window of plugins - use KDE standard icons in object lists (need to be replaced by customized icons in the long run) - don't try to show higher numeric precision than is actually available - fix bug in scatterplot plugin - remove configure options --with-r-share and --with-r-doc (obsoleted by improved wrapper script) - allow deletion of several rows at once in the data.frame editor and fix some bugs regarding deletion of rows - prevent R-Object listviews (Varselector, Workspace Browser) from growing excessively wide - make Ctrl+V work in the console - middle mouse pastes selection, not global clipboard in the console, as expected - clean up output of require (quietly=FALSE) - some fixes for R 2.3.0 - pressing Cancel in readline ("R backend requests information") dialog interrupts the command - process R X11 events while modal dialogs (such as "R backend requests information") are shown - allow empty input in readline ("R backend requests information") dialog - make Ctrl+C/copy work in output and help windows --- Version 0.3.5 - Apr-16-2006 - give back focus to the script editor after running line/selection/all - reworded exit dialog box - show info message, if retrieving context help fails - pressing Ctrl+C in the console resets a partial command, if any - added syntax highlighting in the console - configure: if not overriden, and $KDEDIR is not set, assume --prefix=`kde-config --prefix` instead of /usr/local/kde/ - remove some GUI clutter for Script File editor - rename "Command File" to "Script File" - rename "Command Stack" window to "Pending Jobs" - backspace or del clear cells in data.frame editor - reworked factor levels editor - in trailing rows/columns of the data.frame editor, change background color, when selected - explicitely set LC_NUMERIC to "C" at startup. R does not work correctly with other settings of LC_NUMERIC - many new plugins by Stefan Roediger - clean up R temporary files on shutdown - when switching between dialog and wizard interfaces, GUI settings are copied - fix crash with objects with quotes in their names - add options to show help-search and R help index to Help menu - complete rework of plugins code - plugins can embed other plugins - code display (for plugins) gets updated reliably - avoid leftover PHP processes if rkward crashes or gets killed - autodetection of R_HOME and related variables during configure - fix status bar - handle non-existing and non-local URLs (links) in help window - use a shell wrapper around rkward to do some setup just like R. This should fix a number of minor quirks - configure: use single configure option for location of R_HOME, and add check for libR.so - yet more code cleanups - keep main window caption up to date - update tab caption for command editor if filename changes (i.e. saved as) - indicate if a document in a command editor is modified (unsaved) --- Version 0.3.4 - Nov-08-2005 - assorted bugfixes - include debian-dir in source distribution - real time display of command output in the console - automatically scroll command log to the bottom when new lines are added - less verbose output in command log (removed "issuing command", etc.) - use KXMLGUI for placing plugins in the menu (mostly an internal change, but menu placement should remain sane, when active window changes, now) - help-, output-, and editor-views can be detached to separate windows - allow to bypass startup dialog - back/forward navigation in help pages - can print help pages and output - better handling of errors while installing packages - limit number of lines to display in console and command log (user setting) - remove menu clutter from help window, and output window - avoid adding incomplete (multi-line) Console commands to the Command Log - Settings->Command Log->show/raise window working again - allow loading/saving of command history (console commands) - add (meaningful) context menu for console - allow interrupting current command running in the console via Ctrl+C - add "Command stack" window for advanced control of R backend (cancel commands, pause chain, view stack) - modifiy rkward.desktop to always start rkward in a terminal. The backend sometimes behaves strangely, if this is not done. - use available.packages instead of CRAN.packages (deprecated in R 2.2.0) - fix installation of standard_plugins.pluginmap - allow configuration of many R options in Settings->R backend - updated French translation - make "up"-button work in R-backend repository settings - more code cleanups --- Version 0.3.3 - Oct-06-2005 - honor debug-level setting everywhere - reworked handling of output from R (no more file-sinks) - fix display of warning messages (behavior does not match plain R, yet, but is mostly sane) - option to change list of package repositories - option to archive downloaded packages - assorted bugfixes - provide context/function help for the console - output-window has been switched to KPart-infrastructure - command-editor has been switched to KPart-infrastructure completely - data-editor uses KPart-infrastructure (better integration) - override standard callbacks used by R for ReadConsole, ShowFiles, EditFile(s), ChooseFile, Suicide, CleanUp. No more reliance on stdin - some cleanups of dead code - plugins can now be placed in different menus - keep list of components/plugins and menu-hierarchy in (few) .pluginmap files instead of parsing directory structure - improved handling of tcltk windows - new icon - decrease default debug level - don't crash when objects change their type from vector to list/data.frame, etc. - graph-creation no longer relies on (removable) R objects (can graph after loading a workspace) --- Version 0.3.2 - Apr-27-2005 - overloaded require works properly with character.only=FALSE - several API documentation improvement - hidden objects get shown/hidden according to configuration setting - settings dialog honors KDE-button order - R's x11 windows now handle their events (x11-device is now usable) - fixed bug in the output (no update on empty output) - paste in the console is properly handled - r console and command log divorced - console can handle incomplete statements (allowing multi-line input) - fixed silly crash on incomplete statements - updated spanish translation - new plugin (graph) - reworked the interface to use kparts properly --- Version 0.3.1 - Mar-26-2005 - added the possibility to unable widget in plugins. - added a new plugin (basic statistics) - added icons in the object browser - using IDEAI dock mode style - new, console-like widget to enter commands manually. - added a class to show html help in an mdi child - added a new dock to search R help - many changes in labels - fixed: bug that prevented RKWard to build on some systems - added R functions to show graphs in the html output and to print objects (using E. Lecoutre's R2HTML) - use a KHTML part to show output (instead of the qt widget) --- Version 0.3.0 - Nov-12-2004 - added french and italian translations - added toolbars and some icons - some small i18n fixes - rework of user interface (now using KMDI and IDEAl mode) - compilation fix for R 2.0 - added basic per variable formatting options in the data editor (alignment, precision) - fixed: cell last edited did not react to single mouse click - fixed: switched to line below after a new row was automatically added - fixed: cursor right did not move to next cell when editing last (active) column - while editing, tab-key moves to next cell --- Version 0.2.9 - Sep-22-2004 - use KDialog::marginHint () and KDialog::spacingHint () in plugins - fixed a crash in the formula widget - converted plugin element to QWidgets (in preparation for more extensive rework of plugin elements) - almost complete rework of data editor - added edit support for factors - added support for editing factor levels - data editor stores data more efficiently - renaming columns in a data.frame does not change their position in the frame - recursively remove children of removed objects (fixes one of probably many memory leaks) - wait for backend-thread to terminate on exit. Avoids crashes when closing RKWard - do not crash if loading the "rkward" R package fails, but rather display a nice error-message --- Version 0.2.8 - Sep-09-2004 - override for require () to show install-packages dialog, if required package is not installed. - Dialog to load/unload packages, update and install packages from CRAN - added capability to cancel commands (not properly integrated in the GUI, yet) - basic name checking for objects added or renamed by the user - remove uneccessary blank are below menu in R-Console window - fix R process busy/idle display - show error message if loading workspace fails - do not ask for save, if workspace is empty - results of user-commands get printed if not invisible (i.e. just like in a regular R session) - provide at least basic feedback on syntax errors - added communication channel from R-backend to RKWard. Initial features: - checking for errors setting up the connection - simple rk.sync for synchronisation of objects modified in the R workspace - rk.get.tempfile.name for generation of a suitable temporary file (e.g. for storing images) - allows for in-place execution of commands or scheduling of commands in the regular stack - auto addition of rows/columns when typing on last row/column - allow deletion of rows without losing meta-data --- Version 0.2.7 - Sep-02-2004 - several compilation fixes for KDE/Qt < 3.2.x (and maybe some new ones were introduced?) - slight reorganization of menus - added menu option to close all editor windows - some improvements in the editing behavior for the table. Some quirks still present - renamed windows - fixed some typos - set initial minimum sizes for object browser and object viewer windows - don't show all windows on startup - ask whether to save current workspace before loading a new one - added command-line option to set, which components to show debug-messages for - added option(s) to show/raise watch-window when new commands get displayed - formula-widget handles multiple table situation correctly - RKVarselector and RKObjectBrowser use a common base class for the list view - RKVarselector and dependent widgets get updated when objects get added/removed/modified - prevent conversion to factors when creating data.frames for now - variables can be added/removed correctly in the data.frame editor. Those changes are propagated to the RObjectBrowser automatically - synchronisation from the GUI to the R-workspace is done immediately - added ability to spawn separate command-editor window(s) with load/save/print/etc. - command-editor part in the watch-window keeps keyboard focus (most of the time) --- Version 0.2.6a - Aug-30-2004 - synchronize edited objects before "View"ing them - fix small typo - quick-and-dirty fix for problem with objects in a new table (e.g. "my.data") never getting synced --- Version 0.2.6 - Aug-29-2004 - added error-handling to workspace saving - ask whether to save workspace on application quit - added menu-entries to clear output-window and watch-log - don't duplicate menu-entries when plugins get re-scanned - place "Analyse"-menu left of help-menu - converted plugins to rely entirely on R for formatting, i.e. the printout ()-section contains pure R-code only - wizard-interface is set as default for plugins - added access to kate-config dialog in the command-editor part of the interface watch - added configuration settings for which types of commands to show in the interface watch - added a nice startup-dialog instead of creating an emtpy table unconditionally - limit size of upper-half in the data.frame editor - added convenience R-library: used to streamline communication with R - fix against data being synced over and over again --- Version 0.2.5 - Aug-24-2004 --- - added simple viewer for R-objects - added simple browser for objects in the R-workspace - allow deletion of columns/objects (finally) - allow editing and using several objects/tables - added command-line parameter "debug-level" - interpret command-line parameter "file" - save and restore size of the main window correctly - use internal representation of object-tree in the R-workspace - store meta-information (labels, etc.) as R-attributes - keep track of whether the windows (watch, output) are open - new method of access to objects in R: getIntVector - use some "global" static pointers instead of passing them around (e.g. for the r-interface) - reorganized sources: now using subdirectories for most components --- Version 0.2.2 - Aug-15-2004 --- - added an improved kate syntax-highlighting definition for R - output-formatting is done in R instead of PHP - added a formula widget for use in plugins (not entirely finished, yet) - fixed some small compilation problems - added spinbox widget for use in plugins - use coloring for "unsatisfied" plugin widgets instead of the "Problems" display - added support for wizard-like plugins - same plugin can be opened several times at once - use katepart for plugin code view, too --- Version 0.2.1 - Aug-08-2004 --- - added "don't show again" setting to initial warning - added correlation matrix plugin - ask for R_HOME-setting if not availalbe at startup - allow direct (read) access of vectors in R - support for selection of multiple variables at once - use a katepart as the command-editor (provides basic syntax-highlighting) - plugins can use a tabbook-layout - display status of R-process in the statusbar - R-backend allows chains of commands that are guaranteed to be run without other intermitting commands - run R in a separate thread --- Version 0.2.0 - Aug-01-2004 --- - Use a PHP-backend for the plugins - Run R embedded (currently synchronous) - Some changes in plugin-creation - Use pretty HTML-output - got rid of ui-files - added descriptive statistics plugin - lots of small changes I didn't document properly --- Version 0.1.0 - Nov-19-2002 --- - Initial features: - A simple backend-interface communicating with R via stdin/stdout - Asynchronous command stack - A simple window for watching communication with R - Ability to issue commands manually - Editor window including meta information on the variables - Copy-and-Paste with some quirks - highlighting of invalid data - load/save of R-workspace (load only works for RKWard data) - Automatic generation of menu-structure and plugin-dialogs from XML-files - Supported plugin widgets: textfield, variable-selector, variable-slot, radiobuttons - independent samples t-test plugin diff --git a/rkward/rbackend/rkbackendtransmitter.cpp b/rkward/rbackend/rkbackendtransmitter.cpp index 62d96049..dc11b1ca 100644 --- a/rkward/rbackend/rkbackendtransmitter.cpp +++ b/rkward/rbackend/rkbackendtransmitter.cpp @@ -1,143 +1,156 @@ /*************************************************************************** rkbackendtransmitter - description ------------------- begin : Thu Nov 18 2010 copyright : (C) 2010, 2013 by Thomas Friedrichsmeier email : thomas.friedrichsmeier@kdemail.net ***************************************************************************/ /*************************************************************************** * * * This program is free software; you can redistribute it and/or modify * * it under the terms of the GNU General Public License as published by * * the Free Software Foundation; either version 2 of the License, or * * (at your option) any later version. * * * ***************************************************************************/ #include "rkbackendtransmitter.h" #include "rkrbackend.h" #include #include "../version.h" #include "../debug.h" RKRBackendTransmitter::RKRBackendTransmitter (const QString &servername, const QString &token) { RK_TRACE (RBACKEND); RKRBackendTransmitter::servername = servername; RKRBackendTransmitter::token = token; } RKRBackendTransmitter::~RKRBackendTransmitter () { RK_TRACE (RBACKEND); if (!current_sync_requests.isEmpty ()) { RK_DEBUG (RBACKEND, DL_WARNING, "%d pending requests while exiting RKRBackendTransmitter", current_sync_requests.size ()); } if (!connection) return; // To prevent closing the process before the frontend has had a chance to see the QuitCommand if (connection->bytesToWrite ()) connection->waitForBytesWritten (1000); msleep (1000); } void RKRBackendTransmitter::timerEvent (QTimerEvent *) { // do not trace flushOutput (false); } void RKRBackendTransmitter::run () { RK_TRACE (RBACKEND); QLocalSocket* con = new QLocalSocket (this); con->connectToServer (servername); setConnection (con); if (!connection->waitForConnected ()) handleTransmissionError ("Could not connect: " + connection->errorString ()); // handshake connection->write (token.toLocal8Bit ().data ()); connection->write ("\n"); connection->write (RKWARD_VERSION); connection->write ("\n"); connection->waitForBytesWritten (); - startTimer (200); // calls flushOutput(false), periodically. See timerEvent() + flushtimerid = startTimer (200); // calls flushOutput(false), periodically. See timerEvent() exec (); } +void RKRBackendTransmitter::doExit(){ + RK_TRACE (RBACKEND); + auto con = connection; + killTimer(flushtimerid); + connection->waitForBytesWritten (1000); + connection = 0; // See handleTransmissionError + RK_DEBUG(RBACKEND, DL_DEBUG, "Aborting connection to frontend"); + con->abort(); // TODO: This never seems to complete! + RK_DEBUG(RBACKEND, DL_DEBUG, "Done aborting connection to frontend"); + exit(0); +} + void RKRBackendTransmitter::writeRequest (RBackendRequest *request) { RK_TRACE (RBACKEND); if (request->type != RBackendRequest::Output) flushOutput (true); transmitRequest (request); connection->flush (); if (request->synchronous) { current_sync_requests.append (request); RK_DEBUG (RBACKEND, DL_DEBUG, "Expecting replies for %d requests (added %p)", current_sync_requests.size (), request); } else { delete request; } } void RKRBackendTransmitter::requestReceived (RBackendRequest* request) { RK_TRACE (RBACKEND); // first check for requests which originated in the frontend if (request->type == RBackendRequest::Interrupt) { RKRBackend::this_pointer->interruptCommand (request->params.value ("commandid", -1).toInt ()); } else if (request->type == RBackendRequest::PriorityCommand) { RKRBackend::this_pointer->setPriorityCommand (request->takeCommand ()); } else { // requests which originated in the backend below this line if (current_sync_requests.isEmpty ()) { RK_ASSERT (false); return; } // "Synchronous" requests are not necessarily answered in the order they have been queued int id = request->id; RBackendRequest* current_sync_request = 0; for (int i = current_sync_requests.size () - 1; i >= 0; --i) { RBackendRequest *candidate = current_sync_requests[i]; if (id == candidate->id) { current_sync_request = current_sync_requests.takeAt (i); break; } } RK_ASSERT (current_sync_request); if (current_sync_request->type == RBackendRequest::Output) { delete current_sync_request; // this was just our internal request } else { current_sync_request->mergeReply (request); current_sync_request->done = true; } RK_DEBUG (RBACKEND, DL_DEBUG, "Expecting replies for %d requests (popped %p)", current_sync_requests.size (), current_sync_request); } delete request; } void RKRBackendTransmitter::flushOutput (bool force) { if (!current_sync_requests.isEmpty ()) return; RKRBackend::this_pointer->fetchStdoutStderr (force); ROutputList out = RKRBackend::this_pointer->flushOutput (force); if (out.isEmpty ()) return; RK_TRACE (RBACKEND); // output request would not strictly need to be synchronous. However, making them synchronous ensures that the frontend is keeping up with the output sent by the backend. RBackendRequest* request = new RBackendRequest (true, RBackendRequest::Output); request->output = new ROutputList (out); writeRequest (request); } void RKRBackendTransmitter::handleTransmissionError (const QString &message) { RK_TRACE (RBACKEND); + if (!connection) return; // regular exit, or we did not even get to the point of setting up the connection. RK_DEBUG (RBACKEND, DL_ERROR, "%s", qPrintable ("Transmission error " + message)); RKRBackend::tryToDoEmergencySave (); } diff --git a/rkward/rbackend/rkbackendtransmitter.h b/rkward/rbackend/rkbackendtransmitter.h index 4539385a..d8fbc446 100644 --- a/rkward/rbackend/rkbackendtransmitter.h +++ b/rkward/rbackend/rkbackendtransmitter.h @@ -1,45 +1,47 @@ /*************************************************************************** rkbackendtransmitter - description ------------------- begin : Thu Nov 18 2010 copyright : (C) 2010 by Thomas Friedrichsmeier email : thomas.friedrichsmeier@kdemail.net ***************************************************************************/ /*************************************************************************** * * * This program is free software; you can redistribute it and/or modify * * it under the terms of the GNU General Public License as published by * * the Free Software Foundation; either version 2 of the License, or * * (at your option) any later version. * * * ***************************************************************************/ #ifndef RKBACKENDTRANSMITTER_H #define RKBACKENDTRANSMITTER_H #include "rktransmitter.h" /** Private class used by the RKRBackendProtocol, in case of the backend running in a split process. This will be used as the secondary thread, and takes care of serializing, sending, receiving, and unserializing requests. */ class RKRBackendTransmitter : public RKAbstractTransmitter { Q_OBJECT public: RKRBackendTransmitter (const QString &servername, const QString &token); ~RKRBackendTransmitter (); void publicmsleep (int delay) { msleep (delay); }; void run () override; + void doExit(); // To be called from main thread via QMetaMethod::invoke void writeRequest (RBackendRequest *request) override; void requestReceived (RBackendRequest *request) override; void handleTransmissionError (const QString &message) override; private: void timerEvent (QTimerEvent *event) override; void flushOutput (bool force); QList current_sync_requests; // pointers to the request that we expect a reply for. Yes, internally, this can be several requests. QString servername; + int flushtimerid; }; #endif diff --git a/rkward/rbackend/rkfrontendtransmitter.cpp b/rkward/rbackend/rkfrontendtransmitter.cpp index 180417dd..3a09db84 100644 --- a/rkward/rbackend/rkfrontendtransmitter.cpp +++ b/rkward/rbackend/rkfrontendtransmitter.cpp @@ -1,238 +1,239 @@ /*************************************************************************** rkfrontendtransmitter - description ------------------- begin : Thu Nov 04 2010 copyright : (C) 2010-2019 by Thomas Friedrichsmeier email : thomas.friedrichsmeier@kdemail.net ***************************************************************************/ /*************************************************************************** * * * This program is free software; you can redistribute it and/or modify * * it under the terms of the GNU General Public License as published by * * the Free Software Foundation; either version 2 of the License, or * * (at your option) any later version. * * * ***************************************************************************/ #include "rkfrontendtransmitter.h" #include "rkrbackendprotocol_frontend.h" #include "rkwarddevice/rkgraphicsdevice_frontendtransmitter.h" #include "../misc/rkcommonfunctions.h" #include "../settings/rksettingsmodulegeneral.h" #include "../rkglobals.h" #include #include #include #include #include #include #include #include #include #include "../version.h" #include "../debug.h" QString findBackendAtPath (const QString &path) { QDir dir (path); dir.makeAbsolute (); #ifdef Q_OS_WIN QString ret = dir.filePath ("rkward.rbackend.exe"); #else QString ret = dir.filePath ("rkward.rbackend"); #endif RK_DEBUG (RBACKEND, DL_DEBUG, "Looking for backend at %s", qPrintable (ret)); QFileInfo fi (ret); if (fi.exists () && fi.isExecutable ()) return ret; return QString (); } RKFrontendTransmitter::RKFrontendTransmitter () : RKAbstractTransmitter () { RK_TRACE (RBACKEND); rkd_transmitter = new RKGraphicsDeviceFrontendTransmitter (); start (); } RKFrontendTransmitter::~RKFrontendTransmitter () { RK_TRACE (RBACKEND); delete rkd_transmitter; RK_ASSERT (!server->isListening ()); } QString localeDir () { // adapted from KCatalog::catalogLocaleDir() QString relpath = QStringLiteral ("%1/LC_MESSAGES/rkward.mo").arg (QLocale ().name ().section ('_', 0, 0)); QString file = QStandardPaths::locate (QStandardPaths::GenericDataLocation, QStringLiteral ("locale/") + relpath); if (file.isEmpty ()) return QString (); return QFileInfo (file.left (file.size() - relpath.size ())).absolutePath (); } void RKFrontendTransmitter::run () { RK_TRACE (RBACKEND); // start server qsrand (QTime::currentTime ().msec ()); // Workaround for some versions of kcoreaddons (5.21.0 through at least 5.34.0). See https://phabricator.kde.org/D5966 server = new QLocalServer (this); // we add a bit of randomness to the servername, as in general the servername must be unique // there could be conflicts with concurrent or with previous crashed rkward sessions. if (!server->listen ("rkward" + KRandom::randomString (8))) handleTransmissionError ("Failure to start frontend server: " + server->errorString ()); connect (server, &QLocalServer::newConnection, this, &RKFrontendTransmitter::connectAndEnterLoop, Qt::QueuedConnection); // start backend backend = new QProcess (this); // Try to synchronize language selection in frontend and backend QStringList env = QProcess::systemEnvironment (); int index = env.indexOf (QRegExp("^LANGUAGE=.*", Qt::CaseInsensitive)); if (index >= 0) env.removeAt (index); env.append ("LANGUAGE=" + QLocale ().name ().section ('_', 0, 0)); backend->setEnvironment (env); QStringList args; args.append ("--debug-level=" + QString::number (RK_Debug::RK_Debug_Level)); // NOTE: QProcess quotes its arguments, *but* properly passing all spaces and quotes through the R CMD wrapper, seems near(?) impossible on Windows. Instead, we use percent encoding, internally. args.append ("--server-name=" + server->fullServerName ().toUtf8 ().toPercentEncoding ()); args.append ("--rkd-server-name=" + rkd_transmitter->serverName ().toUtf8 ().toPercentEncoding ()); args.append ("--data-dir=" + RKSettingsModuleGeneral::filesPath ().toUtf8 ().toPercentEncoding ()); args.append ("--locale-dir=" + localeDir ().toUtf8 ().toPercentEncoding ()); connect (backend, static_cast(&QProcess::finished), this, &RKFrontendTransmitter::backendExit); QString backend_executable = findBackendAtPath (QCoreApplication::applicationDirPath ()); #ifdef Q_OS_MACOS if (backend_executable.isEmpty ()) backend_executable = findBackendAtPath (QCoreApplication::applicationDirPath () + "../Resources"); // an appropriate location in a standalone app-bundle #endif if (backend_executable.isEmpty ()) backend_executable = findBackendAtPath (QCoreApplication::applicationDirPath () + "/rbackend"); // for running directly from the build-dir if (backend_executable.isEmpty ()) backend_executable = findBackendAtPath (QCoreApplication::applicationDirPath () + "../lib/libexec"); #ifdef Q_OS_MACOS if (backend_executable.isEmpty ()) backend_executable = findBackendAtPath (QCoreApplication::applicationDirPath () + "/../../../rbackend"); if (backend_executable.isEmpty ()) backend_executable = findBackendAtPath (QCoreApplication::applicationDirPath () + "/../Frameworks/libexec"); // For running from .dmg created by craft --package rkward #endif if (backend_executable.isEmpty ()) backend_executable = findBackendAtPath (RKWARD_BACKEND_PATH); if (backend_executable.isEmpty ()) handleTransmissionError (i18n ("The backend executable could not be found. This is likely to be a problem with your installation.")); QString debugger = RKGlobals::startup_options["backend-debugger"].toString (); args.prepend (RKCommonFunctions::windowsShellScriptSafeCommand (backend_executable)); if (!debugger.isEmpty ()) { args = debugger.split (' ') + args; } #ifdef Q_OS_MACOS // Resolving libR.dylib and friends is a pain on MacOS, and running through R CMD does not always seem to be enough. // (Apparently DYLIB_FALLBACK_LIBRARY_PATH is ignored on newer versions of MacOS). Safest best seems to be to start in the lib directory, itself. QProcess dummy; dummy.start (qgetenv ("R_BINARY"), QStringList() << "--slave" << "--no-save" << "--no-init-file" << "-e" << "cat(R.home('lib'))"); dummy.waitForFinished (); QString r_home = QString::fromLocal8Bit (dummy.readAllStandardOutput ()); RK_DEBUG(RBACKEND, DL_INFO, "Setting working directory to %s", qPrintable (r_home)); backend->setWorkingDirectory (r_home); #endif args.prepend ("CMD"); if (DL_DEBUG >= RK_Debug::RK_Debug_Level) { qDebug ("%s", qPrintable (qgetenv ("R_BINARY"))); qDebug ("%s", qPrintable (args.join ("\n"))); } backend->start (qgetenv ("R_BINARY"), args, QIODevice::ReadOnly); if (!backend->waitForStarted ()) { handleTransmissionError (i18n ("The backend executable could not be started. Error message was: %1", backend->errorString ())); } waitForCanReadLine (backend, 3000); token = QString::fromLocal8Bit (backend->readLine ()).trimmed (); backend->closeReadChannel (QProcess::StandardError); backend->closeReadChannel (QProcess::StandardOutput); exec (); - backend->waitForFinished (); + // It's ok to only give backend a short time to finish. We only get here, after QuitCommand has been handled by the backend + backend->waitForFinished(1000); if (!connection) { RK_ASSERT (false); return; } } void RKFrontendTransmitter::waitForCanReadLine (QIODevice* con, int msecs) { RK_TRACE (RBACKEND); // NOTE: On Qt5+Windows, readyReady may actually come in char by char, so calling waitForReadyRead() does not guarantee we will // see the full line, at all. But also, of course, we want to put some cap on trying. Using a time threshold for this. QTime time; time.start (); while ((!con->canReadLine ()) && (time.elapsed () < msecs)) con->waitForReadyRead (500); } void RKFrontendTransmitter::connectAndEnterLoop () { RK_TRACE (RBACKEND); RK_ASSERT (server->hasPendingConnections ()); QLocalSocket *con = server->nextPendingConnection (); server->close (); // handshake waitForCanReadLine (con, 1000); QString token_c = QString::fromLocal8Bit (con->readLine ()). trimmed (); if (token_c != token) handleTransmissionError (i18n ("Error during handshake with backend process. Expected token '%1', received token '%2'", token, token_c)); waitForCanReadLine (con, 1000); QString version_c = QString::fromLocal8Bit (con->readLine ().trimmed ()); if (version_c != RKWARD_VERSION) handleTransmissionError (i18n ("Version mismatch during handshake with backend process. Frontend is version '%1' while backend is '%2'.\nPlease fix your installation.", QString (RKWARD_VERSION), version_c)); setConnection (con); } void RKFrontendTransmitter::requestReceived (RBackendRequest* request) { RK_TRACE (RBACKEND); if (request->type == RBackendRequest::Output) { ROutputList* list = request->output; for (int i = 0; i < list->size (); ++i) { ROutput *out = (*list)[i]; if (handleOutput (out->output, out->output.length (), out->type)) { RKRBackendEvent* event = new RKRBackendEvent (new RBackendRequest (false, RBackendRequest::OutputStartedNotification)); qApp->postEvent (RKRBackendProtocolFrontend::instance (), event); } delete (out); } delete list; request->output = 0; RK_ASSERT (request->synchronous); writeRequest (request); // to tell the backend, that we are keeping up. Also deletes the request. return; } RKRBackendEvent* event = new RKRBackendEvent (request); qApp->postEvent (RKRBackendProtocolFrontend::instance (), event); } void RKFrontendTransmitter::backendExit (int exitcode) { RK_TRACE (RBACKEND); if (!exitcode && token.isEmpty ()) handleTransmissionError (i18n ("The backend process could not be started. Please check your installation.")); else if (token.isEmpty ()) handleTransmissionError (i18n ("The backend process failed to start with exit code %1.", exitcode)); else handleTransmissionError (i18n ("Backend process has exited with code %1.", exitcode)); } void RKFrontendTransmitter::writeRequest (RBackendRequest *request) { RK_TRACE (RBACKEND); transmitRequest (request); connection->flush (); delete request; } void RKFrontendTransmitter::handleTransmissionError (const QString &message) { RK_TRACE (RBACKEND); if (connection) connection->close (); RBackendRequest* req = new RBackendRequest (false, RBackendRequest::BackendExit); req->params["message"] = message; RKRBackendEvent* event = new RKRBackendEvent (req); qApp->postEvent (RKRBackendProtocolFrontend::instance (), event); exit (); } diff --git a/rkward/rbackend/rkrbackend.cpp b/rkward/rbackend/rkrbackend.cpp index d97bcb5b..c84903e9 100644 --- a/rkward/rbackend/rkrbackend.cpp +++ b/rkward/rbackend/rkrbackend.cpp @@ -1,1765 +1,1767 @@ /*************************************************************************** rkrbackend - description ------------------- begin : Sun Jul 25 2004 copyright : (C) 2004 - 2019 by Thomas Friedrichsmeier email : thomas.friedrichsmeier@kdemail.net ***************************************************************************/ /*************************************************************************** * * * This program is free software; you can redistribute it and/or modify * * it under the terms of the GNU General Public License as published by * * the Free Software Foundation; either version 2 of the License, or * * (at your option) any later version. * * * ***************************************************************************/ #include "rkrbackend.h" #ifdef Q_OS_WIN # include # include # undef ERROR // clashes with R # define Win32 // needed for R includes #else # include # include # include #endif // statics RKRBackend *RKRBackend::this_pointer = 0; RKRBackend::RKReplStatus RKRBackend::repl_status = { QByteArray (), 0, true, 0, 0, RKRBackend::RKReplStatus::NoUserCommand, 0, RKRBackend::RKReplStatus::NotInBrowserContext, false }; void* RKRBackend::default_global_context = 0; #include #include #include #include #include #include "../core/robject.h" #include "../version.h" #include "../debug.h" #include "rkrsupport.h" #include "rkstructuregetter.h" #include "rksignalsupport.h" #include "rkreventloop.h" #include "../misc/rkcommonfunctions.h" #include #include #include #include #define R_INTERFACE_PTRS 1 // for R_CStackStart/Limit #define CSTACK_DEFNS 1 // keep R from defining tons of aliases #define R_NO_REMAP 1 // What the...? "Conflicting definitions" between stdint.h and Rinterface.h despite the #if in Rinterface.h #define uintptr_t uintptr_t // needed to detect CHARSXP encoding #define IS_UTF8(x) (Rf_getCharCE(x) == CE_UTF8) #define IS_LATIN1(x) (Rf_getCharCE(x) == CE_LATIN1) #include #include #include #include #include #include #include #include #ifdef Q_OS_WIN # include # include # include structRstart RK_R_Params; extern "C" { // why oh why isn't Rinterface.h available on Windows? LibExtern void* R_GlobalContext; LibExtern uintptr_t R_CStackLimit; LibExtern void R_SaveGlobalEnvToFile(char*); } #else # include #endif #ifndef Q_OS_WIN # include // needed for pthread_kill # include // seems to be needed at least on FreeBSD # include // for non-blocking pipes # include #endif ///// i18n #if (defined ENABLE_NLS) // ENABLE_NLS is from Rconfig.h # include # define RK_MSG_DOMAIN "rkward" // define i18n to be compatible with the easiest usage in KDE # define i18n(msgid) QString::fromUtf8(dgettext(RK_MSG_DOMAIN, msgid)) void RK_setupGettext (const char* locale_dir) { bindtextdomain (RK_MSG_DOMAIN, locale_dir); bind_textdomain_codeset (RK_MSG_DOMAIN, "UTF-8"); } #else # define i18n(msgid) QString(msgid) void RK_setupGettext (const char*) {}; #endif ///// interrupting R void RK_scheduleIntr () { RK_DEBUG (RBACKEND, DL_DEBUG, "interrupt scheduled"); RKRBackend::repl_status.interrupted = true; #ifdef Q_OS_WIN UserBreak = 1; #else RKSignalSupport::callOldSigIntHandler (); #endif } void RK_doIntr () { RK_scheduleIntr (); R_CheckUserInterrupt (); } void RKRBackend::scheduleInterrupt () { if (RKRBackendProtocolBackend::inRThread ()) { RK_scheduleIntr (); } else { #ifdef Q_OS_WIN RK_scheduleIntr (); // Thread-safe on windows?! #else pthread_kill ((pthread_t) RKRBackendProtocolBackend::instance ()->r_thread_id, SIGUSR1); // NOTE: SIGUSR1 relays to SIGINT #endif } } void RKRBackend::interruptCommand (int command_id) { RK_TRACE (RBACKEND); QMutexLocker lock (&all_current_commands_mutex); if (all_current_commands.isEmpty ()) return; if ((command_id == -1) || (all_current_commands.last ()->id == command_id)) { if (!too_late_to_interrupt) { RK_DEBUG (RBACKEND, DL_DEBUG, "scheduling interrupt for command id %d", command_id); scheduleInterrupt (); } } else { // if the command to cancel is *not* the topmost command, then do not interrupt, yet. foreach (RCommandProxy *candidate, all_current_commands) { if (candidate->id == command_id) { if (!current_commands_to_cancel.contains (candidate)) { RK_DEBUG (RBACKEND, DL_DEBUG, "scheduling delayed interrupt for command id %d", command_id); current_commands_to_cancel.append (candidate); } } } } } void clearPendingInterrupt_Worker (void *) { R_CheckUserInterrupt (); } void RKRBackend::clearPendingInterrupt () { RK_TRACE (RBACKEND); bool passed = R_ToplevelExec (clearPendingInterrupt_Worker, 0); if (!passed) RK_DEBUG (RBACKEND, DL_DEBUG, "pending interrupt cleared"); } // some functions we need that are not declared #if R_VERSION < R_Version(2,13,0) LibExtern void Rf_PrintWarnings (void); #endif extern "C" void run_Rmainloop (void); #ifndef Q_OS_WIN # include #endif #include "rdata.h" extern SEXP RKWard_RData_Tag; // ############## R Standard callback overrides BEGIN #################### Rboolean RKToplevelStatementFinishedCallback (SEXP expr, SEXP value, Rboolean succeeded, Rboolean visible, void *) { RK_TRACE (RBACKEND); Q_UNUSED (expr); Q_UNUSED (value); Q_UNUSED (visible); if ((RKRBackend::repl_status.eval_depth == 0) && (!RKRBackend::repl_status.browser_context)) { // Yes, toplevel-handlers _do_ get called in a browser context! RK_ASSERT (RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::UserCommandRunning); if (succeeded) { RKRBackend::repl_status.user_command_successful_up_to = RKRBackend::repl_status.user_command_parsed_up_to; if (RKRBackend::repl_status.user_command_completely_transmitted) { RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::NoUserCommand; RKRBackend::this_pointer->commandFinished (); } else RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::UserCommandTransmitted; } else { // well, this point of code is never reached with R up to 2.12.0. Instead failed user commands are handled in doError(). RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::UserCommandFailed; } } return (Rboolean) true; } void RKInsertToplevelStatementFinishedCallback (void *) { RK_TRACE (RBACKEND); if (RKRBackend::this_pointer->r_running) { int pos; Rf_addTaskCallback (&RKToplevelStatementFinishedCallback, 0, &RKInsertToplevelStatementFinishedCallback, "_rkward_main_callback", &pos); } } void RKTransmitNextUserCommandChunk (unsigned char* buf, int buflen) { RK_TRACE (RBACKEND); RK_ASSERT (RKRBackend::repl_status.user_command_transmitted_up_to <= RKRBackend::repl_status.user_command_buffer.length ()); // NOTE: QByteArray::length () does not count the trailing '\0' const char* current_buffer = RKRBackend::repl_status.user_command_buffer.data (); current_buffer += RKRBackend::repl_status.user_command_transmitted_up_to; // Skip what we have already transmitted bool reached_eof = false; int pos = 0; const int max_pos = buflen - 2; // one for the termination bool reached_newline = false; while (true) { buf[pos] = *current_buffer; if (*current_buffer == '\n') { reached_newline = true; break; } else if (*current_buffer == ';') break; else if (*current_buffer == '\0') { reached_eof = true; break; } if (pos >= max_pos) break; ++current_buffer; ++pos; } RKRBackend::repl_status.user_command_transmitted_up_to += (pos + 1); if (reached_eof) { buf[pos] = '\n'; RKRBackend::repl_status.user_command_completely_transmitted = true; } buf[++pos] = '\0'; if (reached_newline || reached_eof) { // Making this request synchronous is a bit painful. However, without this, it's extremely difficult to get correct interleaving of output and command lines RBackendRequest req (true, RBackendRequest::CommandLineIn); req.params["commandid"] = RKRBackend::this_pointer->current_command->id; RKRBackend::this_pointer->handleRequest (&req); } } // forward declaration needed on Windows void RCleanUp (SA_TYPE saveact, int status, int RunLast); int RReadConsole (const char* prompt, unsigned char* buf, int buflen, int hist) { RK_TRACE (RBACKEND); RK_ASSERT (buf && buflen); RK_ASSERT (RKRBackend::repl_status.eval_depth >= 0); if (RKRBackend::repl_status.browser_context) { // previously we were in a browser context. Check, whether we've left that. if (RKRBackend::default_global_context == R_GlobalContext) { RKRBackend::repl_status.browser_context = RKRBackend::RKReplStatus::NotInBrowserContext; RKRBackend::this_pointer->handlePlainGenericRequest (QStringList ("endBrowserContext"), false); } } if ((!RKRBackend::repl_status.browser_context) && (RKRBackend::repl_status.eval_depth == 0)) { while (1) { if (RKRBackend::repl_status.user_command_status == RKRBackend::RKReplStatus::NoUserCommand) { RCommandProxy *command = RKRBackend::this_pointer->fetchNextCommand (); if (!command) { #ifdef Q_OS_WIN // Can't easily override R_CleanUp on Windows, so we're calling it manually, here, then force exit if (RKRBackend::this_pointer->killed == RKRBackend::ExitNow) RCleanUp (SA_NOSAVE, 0, 0); else RCleanUp (SA_SUICIDE, 1, 0); exit (0); #endif return 0; // jumps out of the event loop! } if (!(command->type & RCommand::User)) { RKRBackend::this_pointer->runCommand (command); RKRBackend::this_pointer->commandFinished (); } else { // so, we are about to transmit a new user command, which is quite a complex endeavor... /* Some words about running user commands: - User commands can only be run at the top level of execution, not in any sub-stacks. But then, they should never get there, in the first place. - Handling user commands is totally different from all other commands, and relies on R's "REPL" (read-evaluate-print-loop). This is a whole bunch of dedicated code, but there is no other way to achieve handling of commands as if they had been entered on a plain R console (including auto-printing, and toplevel handlers). Most importantly, since important symbols are not exported, such as R_Visible. Vice versa, it is not possible to treat all commands like user commands, esp. in substacks. Problems to deal with: - R_ReadConsole serves a lot of different functions, including reading in code, but also handling user input for readline() or browser(). This makes it necessary to carefully track the current status using "repl_status". You will find repl_status to be modified at a couple of different functions. - One difficulty lies in finding out, just when a command has finished (successfully or with an error). RKToplevelStatementFinishCallback(), and doError() handle the respective cases. NOTE; in R 2.12.0 and above, Rf_countContexts() might help to find out when we are back to square 1! */ RKRBackend::repl_status.user_command_transmitted_up_to = 0; RKRBackend::repl_status.user_command_completely_transmitted = false; RKRBackend::repl_status.user_command_parsed_up_to = 0; RKRBackend::repl_status.user_command_successful_up_to = 0; RKRBackend::repl_status.user_command_buffer = RKRBackend::fromUtf8 (command->command); RKTransmitNextUserCommandChunk (buf, buflen); RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::UserCommandTransmitted; return 1; } } else if (RKRBackend::repl_status.user_command_status == RKRBackend::RKReplStatus::UserCommandTransmitted) { if (RKRBackend::repl_status.user_command_completely_transmitted) { // fully transmitted, but R is still asking for more? This looks like an incomplete statement. // HOWEVER: It may also have been an empty statement such as " ", so let's check whether the prompt looks like a "continue" prompt bool incomplete = false; if (RKRBackend::toUtf8 (prompt) == RKRSupport::SEXPToString (Rf_GetOption (Rf_install ("continue"), R_BaseEnv))) { incomplete = true; } if (incomplete) RKRBackend::this_pointer->current_command->status |= RCommand::Failed | RCommand::ErrorIncomplete; RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::ReplIterationKilled; if (RKRBackend::repl_status.user_command_parsed_up_to <= 0) RKRBackend::this_pointer->startOutputCapture (); // HACK: No capture active, but commandFinished() will try to end one Rf_error (""); // to discard the buffer } else { RKTransmitNextUserCommandChunk (buf, buflen); return 1; } } else if (RKRBackend::repl_status.user_command_status == RKRBackend::RKReplStatus::UserCommandSyntaxError) { RKRBackend::this_pointer->current_command->status |= RCommand::Failed | RCommand::ErrorSyntax; RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::NoUserCommand; if (RKRBackend::repl_status.user_command_parsed_up_to <= 0) RKRBackend::this_pointer->startOutputCapture (); // HACK: No capture active, but commandFinished() will try to end one RKRBackend::this_pointer->commandFinished (); } else if (RKRBackend::repl_status.user_command_status == RKRBackend::RKReplStatus::UserCommandRunning) { // This can mean three different things: // 1) User called readline () // 2) User called browser () // 3) R jumped us back to toplevel behind our backs. // Let's find out, which one it is. if (hist && (RKRBackend::default_global_context != R_GlobalContext)) { break; // this looks like a call to browser(). Will be handled below. } int n_frames = 0; RCommandProxy *dummy = RKRBackend::this_pointer->runDirectCommand ("sys.nframe()", RCommand::GetIntVector); if ((dummy->getDataType () == RData::IntVector) && (dummy->getDataLength () == 1)) { n_frames = dummy->intVector ().at (0); } // What the ??? Why does this simple version always return 0? //int n_frames = RKRSupport::SEXPToInt (RKRSupport::callSimpleFun0 (Rf_install ("sys.nframe"), R_GlobalEnv)); if (n_frames < 1) { // No active frames? This can't be a call to readline(), then, so probably R jumped us back to toplevel, behind our backs. // For safety, let's reset and start over. RKRBackend::this_pointer->current_command->status |= RCommand::Failed | RCommand::ErrorOther; RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::ReplIterationKilled; Rf_error(""); // to discard the buffer } else { // A call to readline(). Will be handled below break; } } else if (RKRBackend::repl_status.user_command_status == RKRBackend::RKReplStatus::UserCommandFailed) { RKRBackend::this_pointer->current_command->status |= RCommand::Failed | RCommand::ErrorOther; RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::NoUserCommand; RKRBackend::this_pointer->commandFinished (); } else { RK_ASSERT (RKRBackend::repl_status.user_command_status == RKRBackend::RKReplStatus::ReplIterationKilled); RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::NoUserCommand; RKRBackend::this_pointer->commandFinished (); } } } // here, we handle readline() calls and such, i.e. not the regular prompt for code // browser() also takes us here. QVariantMap params; RBackendRequest::RCallbackType request_type = RBackendRequest::ReadLine; params["prompt"] = QVariant (prompt); params["cancelled"] = QVariant (false); // add info for browser requests if (hist && (RKRBackend::default_global_context != R_GlobalContext)) { if (RKRBackend::repl_status.browser_context == RKRBackend::RKReplStatus::InBrowserContextPreventRecursion) { qstrncpy ((char *) buf, "n\n", buflen); // skip this, by feeding the browser() a continue return 1; } else { RKRBackend::repl_status.browser_context = RKRBackend::RKReplStatus::InBrowserContextPreventRecursion; RCommandProxy *dummy = RKRBackend::this_pointer->runDirectCommand (".rk.callstack.info()", RCommand::GetStructuredData); request_type = RBackendRequest::Debugger; if ((dummy->getDataType () == RData::StructureVector) && (dummy->getDataLength () >= 4)) { RData::RDataStorage dummy_data = dummy->structureVector (); params["calls"] = QVariant (dummy_data.at (0)->stringVector ()); params["funs"] = QVariant (dummy_data.at (1)->stringVector ()); params["envs"] = QVariant (dummy_data.at (2)->stringVector ()); params["locals"] = QVariant (dummy_data.at (3)->stringVector ()); params["relsrclines"] = QVariant (dummy_data.at (4)->stringVector ()); // hacky: passing a QList is not supported by QVariant } else { RK_ASSERT (false); } RKRBackend::repl_status.browser_context = RKRBackend::RKReplStatus::InBrowserContext; } RK_ASSERT (RKRBackend::repl_status.browser_context == RKRBackend::RKReplStatus::InBrowserContext); } RBackendRequest request (true, request_type); request.params = params; RKRBackend::this_pointer->handleRequest (&request); if (request.params["cancelled"].toBool ()) { if (RKRBackend::this_pointer->current_command) RKRBackend::this_pointer->current_command->status |= RCommand::Canceled; Rf_error ("cancelled"); RK_ASSERT (false); // should not reach this point. } QByteArray localres = RKRBackend::fromUtf8 (request.params["result"].toString ()); // need to append a newline, here. TODO: theoretically, RReadConsole comes back for more, if \0 was encountered before \n. qstrncpy ((char *) buf, localres.left (buflen - 2).append ('\n').data (), buflen); return 1; } #ifdef Q_OS_WIN int RReadConsoleWin (const char* prompt, char* buf, int buflen, int hist) { return RReadConsole (prompt, (unsigned char*) buf, buflen, hist); } #endif bool RKRBackend::fetchStdoutStderr (bool forcibly) { #ifndef Q_OS_WIN if (!forcibly) { if (!stdout_stderr_mutex.tryLock ()) return false; } else { stdout_stderr_mutex.lock (); } if (stdout_stderr_fd < 0) { stdout_stderr_mutex.unlock (); return false; } // it seems, setting this only once is not always enough. fcntl (stdout_stderr_fd, F_SETFL, fcntl (stdout_stderr_fd, F_GETFL, 0) | O_NONBLOCK); char buffer[1024]; while (true) { int bytes = read (stdout_stderr_fd, buffer, 1023); if (bytes <= 0) break; buffer[bytes] = '\0'; // NOTE: we must not risk blocking inside handleOutput, while the stdout_stderr_mutex is locked! handleOutput (RKRBackend::toUtf8 (buffer), bytes, ROutput::Warning, false); } stdout_stderr_mutex.unlock (); #endif return true; } #ifdef Q_OS_WIN bool win_do_detect_winutf8markers = false; QByteArray winutf8start, winutf8stop; #endif void RWriteConsoleEx (const char *buf, int buflen, int type) { RK_TRACE (RBACKEND); RK_DEBUG (RBACKEND, DL_DEBUG, "raw output type %d, size %d: %s", type, buflen, buf); #ifdef Q_OS_WIN // Since R 3.5.0, R on Windows (in CharacterMode == RGui) will print "UTF8 markers" around utf8-encoded sub-sections of the output. // Of course, the actual markers used are not accessible in public API... // So here we try to detect the markers (if any) from print("X", print.gap=1, quote=FALSE), i.e. an expected output of the form // [1] _s_X_e_ // Where _s_ and _e_ are the start and stop markers, respectively. if (win_do_detect_winutf8markers) { QByteArray str(buf, buflen); if (!str.contains('X')) return; // May happen. We better don't rely on how exactly the output is chunked // The value may or may not be printed in the same chunk as the row number, and the following value // so split into whatever values have arrived in this chunk, then pick the one with the 'X' QList candidates = str.split(' '); for (int i = 0; i < candidates.size(); ++i) { if (candidates[i].contains('X')) str = candidates[i]; } winutf8start = str.split('X').value(0); winutf8stop = str.split('X').value(1); return; } #endif // output while nothing else is running (including handlers?) -> This may be a syntax error. if ((RKRBackend::repl_status.eval_depth == 0) && (!RKRBackend::repl_status.browser_context) && (!RKRBackend::this_pointer->isKilled ())) { if (RKRBackend::repl_status.user_command_status == RKRBackend::RKReplStatus::UserCommandTransmitted) { // status UserCommandTransmitted might have been set from RKToplevelStatementFinishedHandler, too, in which case all is fine // (we're probably inside another task handler at this point, then) if (RKRBackend::repl_status.user_command_parsed_up_to < RKRBackend::repl_status.user_command_transmitted_up_to) { RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::UserCommandSyntaxError; } } else if (RKRBackend::repl_status.user_command_status == RKRBackend::RKReplStatus::ReplIterationKilled) { // purge superfluous newlines and empty output return; } else { RK_ASSERT (RKRBackend::repl_status.user_command_status != RKRBackend::RKReplStatus::NoUserCommand); } } if (RKRBackend::this_pointer->killed == RKRBackend::AlreadyDead) return; // this check is mostly for fork()ed clients if (RKRBackend::repl_status.browser_context == RKRBackend::RKReplStatus::InBrowserContextPreventRecursion) return; RKRBackend::this_pointer->fetchStdoutStderr (true); #ifdef Q_OS_WIN // See note, above. Here we handle the UTF8 markers in the output QByteArray str(buf, buflen); QString utf8; if (winutf8start.isEmpty()) { utf8 = RKRBackend::toUtf8 (buf); } else { int pos = 0; while (pos < buflen) { int start = str.indexOf(winutf8start, pos); if (start < 0) { utf8.append (RKRBackend::toUtf8 (str.mid(pos))); break; } utf8.append (RKRBackend::toUtf8 (str.left(start))); start += winutf8start.length(); if (start >= buflen) break; int end = str.indexOf(winutf8stop, start); if (end >= 0) { utf8.append(QString::fromUtf8(str.mid(start, end - start))); pos = end + winutf8stop.length(); } else { utf8.append(QString::fromUtf8(str.mid(start))); break; } } } #else QString utf8 = RKRBackend::toUtf8 (buf); #endif RKRBackend::this_pointer->handleOutput (utf8, buflen, type == 0 ? ROutput::Output : ROutput::Warning); } /** For R callbacks that we want to disable, entirely */ void RDoNothing () { //RK_TRACE (RBACKEND); } void RCleanUp (SA_TYPE saveact, int status, int RunLast) { RK_TRACE (RBACKEND); Q_UNUSED (RunLast); // R_dot_Last is called while "running" the QuitCommand if (RKRBackend::this_pointer->killed == RKRBackend::AlreadyDead) return; // Nothing to clean up if (!RKRBackend::this_pointer->r_running) return; // prevent recursion (if an error occurs, here, we get jumped to the console repl, again!) R_CheckUserInterrupt (); // if there are any user interrupts pendinging, we want them handled *NOW* RKRBackend::this_pointer->r_running = false; // we could be in a signal handler, and the stack base may have changed. uintptr_t old_lim = R_CStackLimit; R_CStackLimit = (uintptr_t)-1; if ((status != 0) && (RKRBackend::this_pointer->killed != RKRBackend::ExitNow)) RKRBackend::this_pointer->killed = RKRBackend::EmergencySaveThenExit; if (RKRBackend::this_pointer->killed == RKRBackend::EmergencySaveThenExit) { if (R_DirtyImage) { QString filename; QDir dir (RKRBackendProtocolBackend::dataDir ()); int i=0; while (true) { filename = "rkward_recover" + QString::number (i) + ".RData"; if (!dir.exists (filename)) break; i++; } filename = dir.absoluteFilePath (filename); R_SaveGlobalEnvToFile (filename.toLocal8Bit ().data ()); qDebug ("Created emergency save file in %s", qPrintable (filename)); } else { qDebug ("Image not dirty while crashing. No emergency save created."); } } if (saveact != SA_SUICIDE) { if (!RKRBackend::this_pointer->isKilled ()) { RBackendRequest request (true, RBackendRequest::BackendExit); request.params["message"] = QVariant (i18n ("The R engine has shut down with status: %1").arg (status)); // krazy:exclude=i18ncheckarg RKRBackend::this_pointer->handleRequest (&request); } R_RunExitFinalizers (); Rf_KillAllDevices (); R_CleanTempDir (); } RKRBackend::this_pointer->killed = RKRBackend::AlreadyDead; // just in case R_CStackLimit = old_lim; // well, it should not matter any longer, but... } void RSuicide (const char* message) { RK_TRACE (RBACKEND); if (!RKRBackend::this_pointer->isKilled ()) { RBackendRequest request (true, RBackendRequest::BackendExit); request.params["message"] = QVariant (i18n ("The R engine has encountered a fatal error:\n%1").arg (message)); // krazy:exclude=i18ncheckarg RKRBackend::this_pointer->handleRequest (&request); RKRBackend::this_pointer->killed = RKRBackend::EmergencySaveThenExit; RCleanUp (SA_SUICIDE, 1, 0); } else { RK_ASSERT (false); } } void RKRBackend::tryToDoEmergencySave () { RK_TRACE (RBACKEND); if (RKRBackendProtocolBackend::inRThread ()) { // If we are in the correct thread, things are easy: RKRBackend::this_pointer->killed = RKRBackend::EmergencySaveThenExit; RCleanUp (SA_SUICIDE, 1, 0); RK_doIntr (); // to jump out of the loop, if needed } else { // If we are in the wrong thread, things are a lot more tricky. We need to cause the R thread to exit, and wait for it to finish saving. // Fortunately, if we are in the wrong thread, that probably means, the R thread did *not* crash, and will thus still be functional this_pointer->killed = EmergencySaveThenExit; + return; RK_scheduleIntr (); for (int i = 0; i < 100; ++i) { // give it up to ten seconds to interrupt and exit the loop if (!this_pointer->r_running) break; RKRBackendProtocolBackend::msleep (100); } if (!this_pointer->r_running) { for (int i = 0; i < 600; ++i) { // give it up to sixty seconds to finish saving if (this_pointer->killed == AlreadyDead) return; // finished RKRBackendProtocolBackend::msleep (100); } } RK_ASSERT (false); // Too bad, but we seem to be stuck. No chance but to return (and crash) } } QStringList charPArrayToQStringList (const char** chars, int count) { QStringList ret; for (int i = 0; i < count; ++i) { // do we need to do locale conversion, here? ret.append (chars[i]); } return ret; } int RChooseFile (int isnew, char *buf, int len) { RK_TRACE (RBACKEND); RBackendRequest request (true, RBackendRequest::ChooseFile); request.params["new"] = QVariant ((bool) isnew); RKRBackend::this_pointer->handleRequest (&request); QByteArray localres = RKRBackend::fromUtf8 (request.params["result"].toString ()); qstrncpy ((char *) buf, localres.data (), len); // return length of filename (strlen (buf)) return (qMin (len - 1, localres.size ())); } /* There are about one million possible entry points to editing / showing files. We try to cover them all, using the following bunch of functions (REditFilesHelper() and doShowEditFiles() are helpers, only) */ void REditFilesHelper (QStringList files, QStringList titles, QString wtitle, RBackendRequest::RCallbackType edit, bool delete_files, bool prompt) { RK_TRACE (RBACKEND); RK_ASSERT ((edit == RBackendRequest::ShowFiles) || (edit == RBackendRequest::EditFiles)); RBackendRequest request (edit != RBackendRequest::ShowFiles, edit); // editing is synchronous, showing is asynchronous if (edit == RBackendRequest::ShowFiles) { request.params["delete"] = QVariant (delete_files); } // see ?file.show() for what appears to be the intended meaning of these first three parameters // (which seem to be inconsistently named even in R itself...) request.params["files"] = QVariant (files); request.params["titles"] = QVariant (titles); request.params["wtitle"] = QVariant (wtitle); request.params["prompt"] = QVariant (prompt); RKRBackend::this_pointer->handleRequest (&request); } int REditFiles (int nfile, const char **file, const char **title, const char *wtitle) { RK_TRACE (RBACKEND); REditFilesHelper (charPArrayToQStringList (file, nfile), charPArrayToQStringList (title, nfile), wtitle, RBackendRequest::EditFiles, false, true); // default implementation seems to return 1 if nfile <= 0, else 1. No idea, what for. see unix/std-sys.c return (nfile <= 0); } SEXP doShowEditFiles (SEXP files, SEXP titles, SEXP wtitle, SEXP del, SEXP prompt, RBackendRequest::RCallbackType edit) { RK_TRACE (RBACKEND); QStringList file_strings = RKRSupport::SEXPToStringList (files); QStringList title_strings = RKRSupport::SEXPToStringList (titles); QString wtitle_string = RKRSupport::SEXPToString (wtitle); bool del_files = RKRSupport::SEXPToInt (del, 0) != 0; bool do_prompt = RKRSupport::SEXPToInt (prompt, 0) != 0; RK_ASSERT (file_strings.size () == title_strings.size ()); RK_ASSERT (file_strings.size () >= 1); REditFilesHelper (file_strings, title_strings, wtitle_string, edit, del_files, do_prompt); return (R_NilValue); } SEXP doEditFiles (SEXP files, SEXP titles, SEXP wtitle, SEXP prompt) { return (doShowEditFiles (files, titles, wtitle, R_NilValue, prompt, RBackendRequest::EditFiles)); } int REditFile (const char *buf) { RK_TRACE (RBACKEND); const char *editor = "none"; const char *title = ""; // does not exist in standard R 2.1.0, so no idea what to return. return REditFiles (1, const_cast (&buf), &title, editor); } SEXP doShowFiles (SEXP files, SEXP titles, SEXP wtitle, SEXP delete_files, SEXP prompt) { return (doShowEditFiles (files, titles, wtitle, delete_files, prompt, RBackendRequest::ShowFiles)); } int RShowFiles (int nfile, const char **file, const char **headers, const char *wtitle, Rboolean del, const char */* pager */) { RK_TRACE (RBACKEND); REditFilesHelper (charPArrayToQStringList (file, nfile), charPArrayToQStringList (headers, nfile), QString (wtitle), RBackendRequest::ShowFiles, (bool) del, true); // default implementation seems to returns 1 on success, 0 on failure. see unix/std-sys.c return 1; } /* FROM R_ext/RStartup.h: "Return value here is expected to be 1 for Yes, -1 for No and 0 for Cancel: symbolic constants in graphapp.h" */ int doDialogHelper (QString caption, QString message, QString button_yes, QString button_no, QString button_cancel, QString default_button, bool wait) { RK_TRACE (RBACKEND); RBackendRequest request (wait, RBackendRequest::ShowMessage); request.params["caption"] = QVariant (caption); request.params["message"] = QVariant (message); request.params["button_yes"] = QVariant (button_yes); request.params["button_no"] = QVariant (button_no); request.params["button_cancel"] = QVariant (button_cancel); request.params["default"] = QVariant (default_button); RKRBackend::this_pointer->handleRequest (&request); if (wait) { QString ret = request.params["result"].toString (); if (ret == "yes") return 1; if (ret == "no") return -1; } return 0; } SEXP doDialog (SEXP caption, SEXP message, SEXP button_yes, SEXP button_no, SEXP button_cancel, SEXP default_button, SEXP wait) { RK_TRACE (RBACKEND); int result = doDialogHelper (RKRSupport::SEXPToString (caption), RKRSupport::SEXPToString (message), RKRSupport::SEXPToString (button_yes), RKRSupport::SEXPToString (button_no), RKRSupport::SEXPToString (button_cancel), RKRSupport::SEXPToString (default_button), RKRSupport::SEXPToInt (wait)); SEXP ret = Rf_allocVector(INTSXP, 1); INTEGER (ret)[0] = result; return ret; } void RShowMessage (const char* message) { RK_TRACE (RBACKEND); doDialogHelper (i18n ("Message from the R backend"), message, "ok", QString (), QString (), "ok", true); } // TODO: currently used on windows, only! int RAskYesNoCancel (const char* message) { RK_TRACE (RBACKEND); if (RKRBackend::this_pointer->killed) return -1; // HACK: At this point R asks whether to save the workspace. We have already handled that. So return -1 for "no" return doDialogHelper (i18n ("Question from the R backend"), message, "yes", "no", "cancel", "yes", true); } void RBusy (int busy) { RK_TRACE (RBACKEND); // R_ReplIteration calls R_Busy (1) after reading in code (if needed), successfully parsing it, and right before evaluating it. if (busy) { if (RKRBackend::repl_status.user_command_status == RKRBackend::RKReplStatus::UserCommandTransmitted) { if (RKRBackend::this_pointer->current_command->type & RCommand::CCOutput) { // flush any previous output capture and start a new one if (RKRBackend::repl_status.user_command_successful_up_to > 0) RKRBackend::this_pointer->printAndClearCapturedMessages (false); RKRBackend::this_pointer->startOutputCapture (); } if (RKRBackend::this_pointer->current_command->type & RCommand::CCCommand) { QByteArray chunk = RKRBackend::repl_status.user_command_buffer.mid (RKRBackend::repl_status.user_command_parsed_up_to, RKRBackend::repl_status.user_command_transmitted_up_to - RKRBackend::repl_status.user_command_parsed_up_to); RKRBackend::this_pointer->printCommand (RKRBackend::toUtf8 (chunk)); } RKRBackend::repl_status.user_command_parsed_up_to = RKRBackend::repl_status.user_command_transmitted_up_to; RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::UserCommandRunning; } } } // ############## R Standard callback overrides END #################### SEXP doUpdateLocale (); // NOTE: stdout_stderr_mutex is recursive to support fork()s, better RKRBackend::RKRBackend () : stdout_stderr_mutex (QMutex::Recursive) { RK_TRACE (RBACKEND); RK_ASSERT (this_pointer == 0); this_pointer = this; current_locale_encoder = 0; // marks locale as not yet initialized doUpdateLocale (); r_running = false; current_command = 0; pending_priority_command = 0; stdout_stderr_fd = -1; } #ifdef Q_OS_WIN void RKRBackend::setupCallbacks () { RK_TRACE (RBACKEND); R_setStartTime(); R_DefParams(&RK_R_Params); // IMPORTANT: see also the #ifndef QS_WS_WIN-portion! RK_R_Params.rhome = get_R_HOME (); RK_R_Params.home = getRUser (); RK_R_Params.CharacterMode = RGui; RK_R_Params.ShowMessage = RShowMessage; RK_R_Params.ReadConsole = RReadConsoleWin; RK_R_Params.WriteConsoleEx = RWriteConsoleEx; RK_R_Params.WriteConsole = 0; RK_R_Params.CallBack = RKREventLoop::winRKEventHandlerWrapper; RK_R_Params.YesNoCancel = RAskYesNoCancel; RK_R_Params.Busy = RBusy; // TODO: callback mechanism(s) for ChosseFile, ShowFiles, EditFiles // TODO: also for RSuicide (Less important, obviously, since this should not be triggered, in normal operation). // NOTE: For RCleanUp see RReadConsole RCleanup? RK_R_Params.R_Quiet = (Rboolean) 0; RK_R_Params.R_Interactive = (Rboolean) 1; } void RKRBackend::connectCallbacks () { RK_TRACE (RBACKEND); R_SetParams(&RK_R_Params); } #else void RKRBackend::setupCallbacks () { RK_TRACE (RBACKEND); } /* SEXP dummyselectlist (SEXP, SEXP, SEXP, SEXP) { qDebug ("got it"); return R_NilValue; }*/ void RKRBackend::connectCallbacks () { RK_TRACE (RBACKEND); // IMPORTANT: see also the #ifdef QS_WS_WIN-portion! // connect R standard callback to our own functions. Important: Don't do so, before our own versions are ready to be used! R_Outputfile = NULL; R_Consolefile = NULL; ptr_R_Suicide = RSuicide; ptr_R_ShowMessage = RShowMessage; // rarely used in R on unix ptr_R_ReadConsole = RReadConsole; ptr_R_WriteConsoleEx = RWriteConsoleEx; ptr_R_WriteConsole = 0; ptr_R_ResetConsole = RDoNothing; ptr_R_FlushConsole = RDoNothing; ptr_R_ClearerrConsole = RDoNothing; ptr_R_Busy = RBusy; ptr_R_CleanUp = RCleanUp; // unfortunately, it seems, we can't safely cancel quitting anymore, here! ptr_R_ShowFiles = RShowFiles; ptr_R_ChooseFile = RChooseFile; // TODO: R devels disabled this for some reason. We set it anyway... ptr_R_EditFile = REditFile; // ptr_R_EditFiles = REditFiles; // undefined reference /* ptr_do_selectlist = dummyselectlist; ptr_do_dataviewer = dummyselectlist;*/ // these two, we won't override // ptr_R_loadhistory = ... // we keep our own history // ptr_R_savehistory = ... // we keep our own history } #endif RKRBackend::~RKRBackend () { RK_TRACE (RBACKEND); } #ifdef _MSC_VER extern "C" int R_interrupts_pending; #else LibExtern int R_interrupts_pending; #endif SEXP doError (SEXP call) { RK_TRACE (RBACKEND); if ((RKRBackend::repl_status.eval_depth == 0) && (!RKRBackend::repl_status.browser_context) && (!RKRBackend::this_pointer->isKilled ()) && (RKRBackend::repl_status.user_command_status != RKRBackend::RKReplStatus::ReplIterationKilled) && (RKRBackend::repl_status.user_command_status != RKRBackend::RKReplStatus::NoUserCommand)) { RKRBackend::repl_status.user_command_status = RKRBackend::RKReplStatus::UserCommandFailed; } if (RKRBackend::repl_status.interrupted) { // it is unlikely, but possible, that an interrupt signal was received, but the current command failed for some other reason, before processing was actually interrupted. In this case, R_interrupts_pending if not yet cleared. // NOTE: if R_interrupts_pending stops being exported one day, we might be able to use R_CheckUserInterrupt() inside an R_ToplevelExec() to find out, whether an interrupt was still pending. if (!R_interrupts_pending) { RKRBackend::repl_status.interrupted = false; if (RKRBackend::repl_status.user_command_status != RKRBackend::RKReplStatus::ReplIterationKilled) { // was interrupted only to step out of the repl iteration QMutexLocker lock (&(RKRBackend::this_pointer->all_current_commands_mutex)); foreach (RCommandProxy *command, RKRBackend::this_pointer->all_current_commands) command->status |= RCommand::Canceled; RK_DEBUG (RBACKEND, DL_DEBUG, "interrupted"); } } } else if (RKRBackend::repl_status.user_command_status != RKRBackend::RKReplStatus::ReplIterationKilled) { QString string = RKRSupport::SEXPToString (call); RKRBackend::this_pointer->handleOutput (string, string.length (), ROutput::Error); RK_DEBUG (RBACKEND, DL_DEBUG, "error '%s'", qPrintable (string)); } return R_NilValue; } SEXP doWs (SEXP name) { if ((!RKRBackend::this_pointer->current_command) || (RKRBackend::this_pointer->current_command->type & RCommand::ObjectListUpdate) || (!(RKRBackend::this_pointer->current_command->type & RCommand::Sync))) { // ignore Sync commands that are not flagged as ObjectListUpdate QString sym = RKRSupport::SEXPToString(name); if (!RKRBackend::this_pointer->changed_symbol_names.contains (sym)) RKRBackend::this_pointer->changed_symbol_names.append (sym); // schedule symbol update for later } return R_NilValue; } SEXP doSubstackCall (SEXP call) { RK_TRACE (RBACKEND); R_CheckUserInterrupt (); QStringList list = RKRSupport::SEXPToStringList (call); /* // this is a useful place to sneak in test code for profiling if (list.value (0) == "testit") { for (int i = 10000; i >= 1; --i) { setWarnOption (i); } return R_NilValue; } */ RKRBackend::this_pointer->handleHistoricalSubstackRequest (list); return R_NilValue; } SEXP doPlainGenericRequest (SEXP call, SEXP synchronous) { RK_TRACE (RBACKEND); R_CheckUserInterrupt (); QStringList ret = RKRBackend::this_pointer->handlePlainGenericRequest (RKRSupport::SEXPToStringList (call), RKRSupport::SEXPToInt (synchronous)); return RKRSupport::StringListToSEXP (ret); } void R_CheckStackWrapper (void *) { R_CheckStack (); } SEXP doUpdateLocale () { RK_TRACE (RBACKEND); RK_DEBUG (RBACKEND, DL_WARNING, "Changing locale"); if (RKRBackend::this_pointer->current_locale_encoder) { delete (RKRBackend::this_pointer->current_locale_encoder); delete (RKRBackend::this_pointer->current_locale_decoder); QTextCodec::setCodecForLocale (0); } RK_ASSERT (QTextCodec::codecForLocale ()); RKRBackend::this_pointer->current_locale_encoder = QTextCodec::codecForLocale ()->makeEncoder (QTextCodec::DefaultConversion); // NOTE: shall pass non-representable characters unmodified, rather than stripping them. RKRBackend::this_pointer->current_locale_decoder = QTextCodec::codecForLocale ()->makeDecoder (QTextCodec::DefaultConversion); RK_DEBUG (RBACKEND, DL_WARNING, "New locale codec is %s", QTextCodec::codecForLocale ()->name ().data ()); return R_NilValue; } // returns the MIME-name of the current locale encoding (from Qt) SEXP doLocaleName () { RK_TRACE (RBACKEND); RK_ASSERT (QTextCodec::codecForLocale()); SEXP res = Rf_allocVector(STRSXP, 1); PROTECT (res); SET_STRING_ELT (res, 0, Rf_mkChar (QTextCodec::codecForLocale()->name ().data ())); UNPROTECT (1); return res; } SEXP doGetStructure (SEXP toplevel, SEXP name, SEXP envlevel, SEXP namespacename) { RK_TRACE (RBACKEND); RKStructureGetter getter (false); RData *ret = getter.getStructure (toplevel, name, envlevel, namespacename); return R_MakeExternalPtr (ret, RKWard_RData_Tag, R_NilValue); } SEXP doGetGlobalEnvStructure (SEXP name, SEXP envlevel, SEXP namespacename) { RK_TRACE (RBACKEND); return doGetStructure (Rf_findVar (Rf_install (CHAR (STRING_ELT (name, 0))), R_GlobalEnv), name, envlevel, namespacename); } /** copy a symbol without touching it (esp. not forcing any promises) */ SEXP doCopyNoEval (SEXP fromname, SEXP fromenv, SEXP toname, SEXP toenv) { RK_TRACE (RBACKEND); if(!Rf_isString (fromname) || Rf_length (fromname) != 1) Rf_error ("fromname is not a single string"); if(!Rf_isString (toname) || Rf_length (toname) != 1) Rf_error ("toname is not a single string"); if(!Rf_isEnvironment (fromenv)) Rf_error ("fromenv is not an environment"); if(!Rf_isEnvironment (toenv)) Rf_error ("toenv is not an environment"); Rf_defineVar (Rf_install (CHAR (STRING_ELT (toname, 0))), Rf_findVar (Rf_install (CHAR (STRING_ELT (fromname, 0))), fromenv), toenv); return (R_NilValue); } SEXP doCaptureOutput (SEXP mode, SEXP capture_messages, SEXP capture_output, SEXP suppress_messages, SEXP suppress_output) { RK_TRACE (RBACKEND); if (RKRSupport::SEXPToInt (mode) == 1) { int cm = 0; if (RKRSupport::SEXPToInt (capture_messages)) cm |= RKROutputBuffer::RecordMessages; if (RKRSupport::SEXPToInt (capture_output)) cm |= RKROutputBuffer::RecordOutput; if (RKRSupport::SEXPToInt (suppress_messages)) cm |= RKROutputBuffer::SuppressMessages; if (RKRSupport::SEXPToInt (suppress_output)) cm |= RKROutputBuffer::SuppressOutput; RKRBackend::this_pointer->pushOutputCapture (cm); return (R_NilValue); } else { return RKRSupport::StringListToSEXP (QStringList (RKRBackend::this_pointer->popOutputCapture (RKRSupport::SEXPToInt (mode) == 2))); } } SEXP RKStartGraphicsDevice (SEXP width, SEXP height, SEXP pointsize, SEXP family, SEXP bg, SEXP title, SEXP antialias); SEXP RKD_AdjustSize (SEXP devnum); SEXP doWs (SEXP name); void doPendingPriorityCommands (); bool RKRBackend::startR () { RK_TRACE (RBACKEND); setupCallbacks (); RKSignalSupport::saveDefaultSignalHandlers (); too_late_to_interrupt = false; r_running = true; int argc = 3; char* argv[3] = { qstrdup ("--slave"), qstrdup ("--no-save"), qstrdup ("--no-restore") }; Rf_initialize_R (argc, argv); #ifdef Q_OS_WIN R_set_command_line_arguments(argc, argv); FlushConsoleInputBuffer(GetStdHandle(STD_INPUT_HANDLE)); #endif #ifndef Q_OS_WIN // re-direct stdout / stderr to a pipe, so we can read output from system() calls int pfd[2]; int error = pipe (pfd); RK_ASSERT (!error); // mostly to silence compile time warning about unused return value dup2 (pfd[1], STDOUT_FILENO); dup2 (pfd[1], STDERR_FILENO); // forward both to a single channel to avoid interleaving hell, for now. close (pfd[1]); stdout_stderr_fd = pfd[0]; #endif #ifndef Q_OS_WIN // It is important to set this *early*, so R does not bail out, if there is an error in .Rprofile. // On windows, set in connectCallbacks() for technical reasons, and that seems sufficient. R_Interactive = (Rboolean) TRUE; #endif setup_Rmainloop (); #ifndef Q_OS_WIN // safety check: If we are beyond the stack boundaries already, we better disable stack checking // this has to come *after* the first setup_Rmainloop ()! Rboolean stack_ok = R_ToplevelExec (R_CheckStackWrapper, (void *) 0); if (!stack_ok) { RK_DEBUG (RBACKEND, DL_WARNING, "R_CheckStack() failed during initialization. Will disable stack checking and try to re-initialize."); RK_DEBUG (RBACKEND, DL_WARNING, "Whether or not things work after this, *please* submit a bug report."); R_CStackStart = (uintptr_t) -1; R_CStackLimit = (uintptr_t) -1; setup_Rmainloop (); } #endif #ifndef Q_OS_WIN // I am not sure, whether it is necessary to repeat this, here. It is not in R 3.0.0. // But historically, it was placed here (after setup_Rmainloop(), and conceivably there // was a reason to that (might have been reset in setup_Rmainloop() in earlier versions // of R. R_Interactive = (Rboolean) TRUE; #endif setlocale (LC_NUMERIC, "C"); // Under some conditions something appears to mess with the locale. R will not work correctly without LC_NUMERIC=C RBackendRequest req (false, RBackendRequest::SetParamsFromBackend); req.params["na_real"] = NA_REAL; // may not be initialized before setup_Rmainloop! req.params["na_int"] = NA_INTEGER; handleRequest (&req); RKWard_RData_Tag = Rf_install ("RKWard_RData_Tag"); RKSignalSupport::installSignalProxies (); // for the crash signals RKSignalSupport::installSigIntAndUsrHandlers (RK_scheduleIntr); // register our functions R_CallMethodDef callMethods [] = { // NOTE: Intermediate cast to void* to avoid compiler warning { "ws", (DL_FUNC) (void*) &doWs, 1 }, { "rk.do.error", (DL_FUNC) (void*) &doError, 1 }, { "rk.do.command", (DL_FUNC) (void*) &doSubstackCall, 1 }, { "rk.do.generic.request", (DL_FUNC) (void*) &doPlainGenericRequest, 2 }, { "rk.get.structure", (DL_FUNC) (void*) &doGetStructure, 4 }, { "rk.get.structure.global", (DL_FUNC) (void*) &doGetGlobalEnvStructure, 3 }, { "rk.copy.no.eval", (DL_FUNC) (void*) &doCopyNoEval, 4 }, { "rk.edit.files", (DL_FUNC) (void*) &doEditFiles, 4 }, { "rk.show.files", (DL_FUNC) (void*) &doShowFiles, 5 }, { "rk.dialog", (DL_FUNC) (void*) &doDialog, 7 }, { "rk.update.locale", (DL_FUNC) (void*) &doUpdateLocale, 0 }, { "rk.locale.name", (DL_FUNC) (void*) &doLocaleName, 0 }, { "rk.capture.output", (DL_FUNC) (void*) &doCaptureOutput, 5 }, { "rk.graphics.device", (DL_FUNC) (void*) &RKStartGraphicsDevice, 7}, { "rk.graphics.device.resize", (DL_FUNC) (void*) &RKD_AdjustSize, 1}, { 0, 0, 0 } }; R_registerRoutines (R_getEmbeddingDllInfo(), NULL, callMethods, NULL, NULL); connectCallbacks(); RKInsertToplevelStatementFinishedCallback (0); RKREventLoop::setRKEventHandler (doPendingPriorityCommands); default_global_context = R_GlobalContext; #ifdef Q_OS_WIN // See the corresponding note in RWriteConsoleEx(). For auto-detecting UTF8 markers in console output. win_do_detect_winutf8markers = true; runDirectCommand("print(c(\"X\",\"Y\"), print.gap=1, quote=FALSE)"); win_do_detect_winutf8markers = false; #endif // What the??? Somehow the first command we run *will* appear to throw a syntax error. Some init step seems to be missing, but where? // Anyway, we just run a dummy to "clear" that trap. runDirectCommand ("\n"); // get info on R runtime version RCommandProxy *dummy = runDirectCommand ("(as.numeric(R.version$major)*1000+as.numeric(R.version$minor)*10)\n", RCommand::GetIntVector); if ((dummy->getDataType () == RData::IntVector) && (dummy->getDataLength () == 1)) { r_version = dummy->intVector ().at (0); } else { RK_ASSERT (false); r_version = 0; } delete dummy; return true; } #ifndef Q_OS_WIN static bool backend_was_forked = false; void prepareFork () { RK_TRACE (RBACKEND); if (!RKRBackendProtocolBackend::inRThread ()) return; // we need to make sure that the transmitter thread does not hold a lock on the mutex! RKRBackend::this_pointer->stdout_stderr_mutex.lock (); } void completeForkMaster () { RK_TRACE (RBACKEND); if (!RKRBackendProtocolBackend::inRThread ()) return; RKRBackend::this_pointer->stdout_stderr_mutex.unlock (); if (backend_was_forked) return; backend_was_forked = true; // Block SIGCHLD in the main thread from now on. I don't fully understand, why, but otherwise, these signals // interrupt the select() call in the fork()ing code of library(parallel) sigset_t new_set; sigemptyset (&new_set); sigaddset (&new_set, SIGCHLD); pthread_sigmask (SIG_BLOCK, &new_set, NULL); // This was used to show a warning message. Unfortunately, however, forks also occur on every popen (i.e. in system(..., intern=TRUE). // RKRBackend::this_pointer->handlePlainGenericRequest (QStringList ("forkNotification"), false); RK_DEBUG (RBACKEND, DL_WARNING, "Backend process forked (for the first time, this session)"); // NOTE: perhaps we can heuristically differentiate from popen by checking sys.calls() for something with "fork" in it. // esp., in case we discover adverse side-effects of blocking SIGCHLD, we should attempt this } void completeForkChild () { RKRBackendProtocolBackend::instance ()->r_thread_id = QThread::currentThreadId(); RKRBackend::this_pointer->killed = RKRBackend::AlreadyDead; // Not quite accurate, but disables all communication with the frontend } #endif void RKRBackend::enterEventLoop () { RK_TRACE (RBACKEND); #ifndef Q_OS_WIN pthread_atfork (prepareFork, completeForkMaster, completeForkChild); #endif run_Rmainloop (); // NOTE: Do NOT run Rf_endEmbeddedR(). It does more that we want. We rely on RCleanup, instead. + RK_DEBUG(RBACKEND, DL_DEBUG, "R loop finished"); } struct SafeParseWrap { SEXP cv; SEXP pr; ParseStatus status; }; void safeParseVector (void *data) { SafeParseWrap *wrap = static_cast (data); wrap->pr = 0; // TODO: Maybe we can use R_ParseGeneral instead. Then we could find the exact character, where parsing fails. Nope: not exported API wrap->pr = R_ParseVector (wrap->cv, -1, &(wrap->status), R_NilValue); } SEXP parseCommand (const QString &command_qstring, RKRBackend::RKWardRError *error) { RK_TRACE (RBACKEND); SafeParseWrap wrap; wrap.status = PARSE_NULL; QByteArray localc = RKRBackend::fromUtf8 (command_qstring); // needed so the string below does not go out of scope const char *command = localc.data (); PROTECT(wrap.cv=Rf_allocVector(STRSXP, 1)); SET_STRING_ELT(wrap.cv, 0, Rf_mkChar(command)); // Yes, if there is an error in the parse, R does jump back to toplevel! // trying to parse list(""=1) is an example in R 3.1.1 R_ToplevelExec (safeParseVector, &wrap); SEXP pr = wrap.pr; UNPROTECT(1); if ((!pr) || (TYPEOF (pr) == NILSXP)) { // got a null SEXP. This means parse was *not* ok, even if R_ParseVector told us otherwise if (wrap.status == PARSE_OK) { wrap.status = PARSE_ERROR; printf ("weird parse error\n"); } } if (wrap.status != PARSE_OK) { if ((wrap.status == PARSE_INCOMPLETE) || (wrap.status == PARSE_EOF)) { *error = RKRBackend::Incomplete; } else if (wrap.status == PARSE_ERROR) { //extern SEXP parseError (SEXP call, int linenum); //parseError (R_NilValue, 0); *error = RKRBackend::SyntaxError; } else { // PARSE_NULL *error = RKRBackend::OtherError; } pr = R_NilValue; } return pr; } SEXP runCommandInternalBase (SEXP pr, RKRBackend::RKWardRError *error) { RK_TRACE (RBACKEND); SEXP exp; int r_error = 0; exp=R_NilValue; if (TYPEOF(pr)==EXPRSXP && LENGTH(pr)>0) { int bi=0; while (bi= RCommand::GetIntVector) && (datatype <= RCommand::GetStructuredData)); RCommandProxy *c = new RCommandProxy (command, RCommand::App | RCommand::Sync | RCommand::Internal | datatype); runCommand (c); return c; } void setWarnOption (int level) { SEXP s, t; PROTECT (t = s = Rf_allocList (2)); SET_TYPEOF (s, LANGSXP); SETCAR (t, Rf_install ("options")); t = CDR (t); SETCAR (t, Rf_ScalarInteger (level)); SET_TAG (t, Rf_install ("warn")); // The above is rougly equivalent to parseCommand ("options(warn=" + QString::number (level) + ")", &error), but ~100 times faster RKRBackend::RKWardRError error; runCommandInternalBase (s, &error); UNPROTECT (1); } void RKRBackend::runCommand (RCommandProxy *command) { RK_TRACE (RBACKEND); RK_ASSERT (command); RKWardRError error = NoError; int ctype = command->type; // easier typing // running user commands is quite different from all other commands and should have been handled by RReadConsole RK_ASSERT (!(ctype & RCommand::User)); if (ctype & RCommand::CCCommand) printCommand (command->command); if (ctype & RCommand::CCOutput) startOutputCapture (); if (ctype & RCommand::QuitCommand) { R_dot_Last (); // should run while communication with frontend is still possible RBackendRequest req (true, RBackendRequest::BackendExit); req.params["regular"] = QVariant (true); handleRequest (&req); killed = ExitNow; } else if (!(ctype & RCommand::EmptyCommand)) { repl_status.eval_depth++; SEXP parsed = parseCommand (command->command, &error); if (error == NoError) { PROTECT (parsed); SEXP exp; #if R_VERSION >= R_Version(2,13,0) int warn_level = RKRSupport::SEXPToInt (Rf_GetOption1 (Rf_install ("warn")), 0); if (warn_level != 1) setWarnOption (1); #endif PROTECT (exp = runCommandInternalBase (parsed, &error)); #if R_VERSION >= R_Version(2,13,0) if (warn_level != 1) setWarnOption (warn_level); #endif if (error == NoError) { if (ctype & RCommand::GetStringVector) { command->setData (RKRSupport::SEXPToStringList (exp)); } else if (ctype & RCommand::GetRealVector) { command->setData (RKRSupport::SEXPToRealArray (exp)); } else if (ctype & RCommand::GetIntVector) { command->setData (RKRSupport::SEXPToIntArray (exp)); } else if (ctype & RCommand::GetStructuredData) { RData *dummy = RKRSupport::SEXPToRData (exp); command->swallowData (*dummy); delete dummy; } } UNPROTECT (2); // exp, parsed } repl_status.eval_depth--; } // common error/status handling if (error != NoError) { command->status |= RCommand::WasTried | RCommand::Failed; if (error == Incomplete) { command->status |= RCommand::ErrorIncomplete; } else if (error == SyntaxError) { command->status |= RCommand::ErrorSyntax; } else if (!(command->status & RCommand::Canceled)) { command->status |= RCommand::ErrorOther; } } else { command->status |= RCommand::WasTried; } } void RKRBackend::setPriorityCommand (RCommandProxy* command) { RK_TRACE (RBACKEND); QMutexLocker lock (&priority_command_mutex); RK_ASSERT (!(command && pending_priority_command)); // for the time being, we support only one priority command at a time pending_priority_command = command; RKREventLoop::wakeRKEventHandler (); } void doPendingPriorityCommands () { RK_TRACE (RBACKEND); if (RKRBackend::this_pointer->killed) return; RCommandProxy *command = RKRBackend::this_pointer->pending_priority_command; RKRBackend::this_pointer->pending_priority_command = 0; if (command) { RK_DEBUG (RBACKEND, DL_DEBUG, "running priority command %s", qPrintable (command->command)); { QMutexLocker lock (&RKRBackend::this_pointer->all_current_commands_mutex); RKRBackend::this_pointer->all_current_commands.append (command); RKRBackend::this_pointer->current_command = command; } RKRBackend::this_pointer->runCommand (command); // TODO: Oh boy, what a mess. Sending notifications should be split from fetchNextCommand() (which is not appropriate, here) RCommandProxy *previous_command_backup = RKRBackend::this_pointer->previous_command; RKRBackend::this_pointer->commandFinished (false); RKRBackend::this_pointer->previous_command = previous_command_backup; RBackendRequest req (false, RBackendRequest::CommandOut); // NOTE: We do *NOT* want a reply to this one, and in particular, we do *NOT* want to do // (recursive) event processing while handling this. req.command = command; RKRBackend::this_pointer->handleRequest (&req); } } // On Windows, using runDirectCommand (".rk.cat.output ...") is not safe during some places where we call this, e.g. in RBusy. // Not a problem on Linux with R 2.13.0, though void RKRBackend::catToOutputFile (const QString &out) { RK_TRACE (RBACKEND); if (output_file.isEmpty ()) { RK_ASSERT (false); return; } QFile f (output_file); if (!f.open (QIODevice::WriteOnly | QIODevice::Append)) { RK_ASSERT (false); return; } f.write (RKRBackend::fromUtf8 (out)); f.close (); } void RKRBackend::printCommand (const QString &command) { RK_TRACE (RBACKEND); QStringList params ("highlightRCode"); params.append (command); QString highlighted = handlePlainGenericRequest (params, true).value (0); catToOutputFile (highlighted); } void RKRBackend::startOutputCapture () { RK_TRACE (RBACKEND); pushOutputCapture (RecordMessages | RecordOutput); } void RKRBackend::printAndClearCapturedMessages (bool with_header) { RK_TRACE (RBACKEND); QString out = popOutputCapture (true); if (out.isEmpty ()) return; if (with_header) out.prepend ("

Messages, warnings, or errors:

\n"); catToOutputFile (out); } void RKRBackend::run (const QString &locale_dir) { RK_TRACE (RBACKEND); killed = NotKilled; previous_command = 0; initialize (QFile::encodeName (locale_dir)); - enterEventLoop (); + enterEventLoop(); } void RKRBackend::commandFinished (bool check_object_updates_needed) { RK_TRACE (RBACKEND); RK_DEBUG (RBACKEND, DL_DEBUG, "done running command %s", qPrintable (current_command->command)); { QMutexLocker lock (&all_current_commands_mutex); too_late_to_interrupt = true; } clearPendingInterrupt (); // Mutex must be unlocked for this! if (current_command->type & RCommand::CCOutput) printAndClearCapturedMessages (current_command->type & RCommand::Plugin); current_command->status -= (current_command->status & RCommand::Running); current_command->status |= RCommand::WasTried; if (current_command->type & RCommand::User) { RK_ASSERT (repl_status.eval_depth == 0); // This method may look a bit over-complex, but remember that repl_status.user_command_successful_up_to works on an *encoded* buffer QByteArray remainder_encoded = repl_status.user_command_buffer.mid (repl_status.user_command_successful_up_to); QString remainder = RKRBackend::toUtf8 (remainder_encoded); current_command->has_been_run_up_to = current_command->command.length () - remainder.length (); } if (check_object_updates_needed || (current_command->type & RCommand::ObjectListUpdate)) { checkObjectUpdatesNeeded (current_command->type & (RCommand::User | RCommand::ObjectListUpdate)); } previous_command = current_command; { QMutexLocker lock (&all_current_commands_mutex); all_current_commands.pop_back(); if (!all_current_commands.isEmpty ()) current_command = all_current_commands.last (); too_late_to_interrupt = false; } } RCommandProxy* RKRBackend::handleRequest (RBackendRequest *request, bool mayHandleSubstack) { RK_TRACE (RBACKEND); RK_ASSERT (request); RKRBackendProtocolBackend::instance ()->sendRequest (request); if ((!request->synchronous) && (!isKilled ())) { RK_ASSERT (mayHandleSubstack); // i.e. not called from fetchNextCommand return 0; } int i = 0; while (!request->done) { if (killed) return 0; // NOTE: processX11Events() may, conceivably, lead to new requests, which may also wait for sub-commands! RKREventLoop::processX11Events (); // NOTE: sleeping and waking up again can be surprisingly CPU-intensive (yes: more than the event processing, above. I have profiled it). // However, we really don't want to introduce too much delay, either. // Thus, the logic is this: If there was no reply within 2 seconds, then probably we're waiting for a user event, and can afford some more // latency (not too much, though, as we still need to process events). if (!request->done) RKRBackendProtocolBackend::msleep (++i < 200 ? 10 : 50); } // TODO remove me? while (pending_priority_command) RKREventLoop::processX11Events (); // Probably not needed, but make sure to process priority commands first at all times. RCommandProxy* command = request->takeCommand (); if (!command) return 0; { QMutexLocker lock (&all_current_commands_mutex); all_current_commands.append (command); current_command = command; } if (!mayHandleSubstack) return command; while (command) { runCommand (command); commandFinished (false); command = fetchNextCommand (); }; { QMutexLocker lock (&all_current_commands_mutex); if (current_commands_to_cancel.contains (current_command)) { RK_DEBUG (RBACKEND, DL_DEBUG, "will now interrupt parent command"); current_commands_to_cancel.removeAll (current_command); scheduleInterrupt (); } } return 0; } RCommandProxy* RKRBackend::fetchNextCommand () { RK_TRACE (RBACKEND); RBackendRequest req (!isKilled (), RBackendRequest::CommandOut); // when killed, we do *not* actually wait for the reply, before the request is deleted. req.command = previous_command; previous_command = 0; return (handleRequest (&req, false)); } void RKRBackend::handleHistoricalSubstackRequest (const QStringList &list) { RK_TRACE (RBACKEND); RBackendRequest request (true, RBackendRequest::HistoricalSubstackRequest); request.params["call"] = list; request.command = current_command; handleRequest (&request); } QStringList RKRBackend::handlePlainGenericRequest (const QStringList ¶meters, bool synchronous) { RK_TRACE (RBACKEND); RBackendRequest request (synchronous, RBackendRequest::PlainGenericRequest); if (parameters.value (0) == "getSessionInfo") { QStringList dummy = parameters; dummy.append (RKRBackendProtocolBackend::backendDebugFile ()); // NOTE: R_SVN_REVISON used to be a string, but has changed to numeric constant in R 3.0.0. QString::arg() handles both. dummy.append (QString (R_MAJOR "." R_MINOR " " R_STATUS " (" R_YEAR "-" R_MONTH "-" R_DAY " r%1)").arg (R_SVN_REVISION)); request.params["call"] = dummy; } else if (parameters.value (0) == "set.output.file") { output_file = parameters.value (1); if (parameters.length () > 2) { RK_ASSERT (parameters.value (2) == "SILENT"); return QStringList (); // For automated testing and previews. The frontend should not be notified, here } request.params["call"] = parameters; } else { request.params["call"] = parameters; } handleRequest (&request); return request.params.value ("return").toStringList (); } void RKRBackend::initialize (const char *locale_dir) { RK_TRACE (RBACKEND); // in RInterface::RInterface() we have created a fake RCommand to capture all the output/errors during startup. Fetch it repl_status.eval_depth++; fetchNextCommand (); startR (); bool lib_load_fail = false; bool sink_fail = false; // Try to load rkward package. If that fails, or is the wrong version, try to install // rkward package, then load again. QString libloc = RKRBackendProtocolBackend::dataDir () + "/.rkward_packages/" + QString::number (r_version / 10); QString versioncheck = QString ("stopifnot(.rk.app.version==\"%1\")\n").arg (RKWARD_VERSION); QString command = "local({\n" " libloc <- " + RKRSharedFunctionality::quote (libloc) + "\n" " if (!dir.exists (libloc)) dir.create(libloc, recursive=TRUE)\n" " ok <- FALSE\n" " suppressWarnings (try ({library (\"rkward\", lib.loc=libloc); " + versioncheck + "; ok <- TRUE}))\n" " if (!ok) {\n" " suppressWarnings (try (detach(\"package:rkward\", unload=TRUE)))\n" " install.packages(normalizePath(paste(libloc, \"..\", c (\"rkward.tgz\", \"rkwardtests.tgz\"), sep=\"/\")), lib=libloc, repos=NULL, type=\"source\")\n" " library (\"rkward\", lib.loc=libloc)\n" " }\n" " .libPaths(c(.libPaths(), libloc))\n" // Add to end search path: Will be avaiable for help serach, but hopefully, not get into the way, otherwise "})\n"; if (!runDirectCommand (command)) lib_load_fail = true; RK_setupGettext (locale_dir); // must happen *after* package loading, since R will re-set it if (!runDirectCommand (versioncheck)) lib_load_fail = true; if (!runDirectCommand (".rk.fix.assignments ()\n")) sink_fail = true; // error/output sink and help browser if (!runDirectCommand ("options (error=quote (.rk.do.error ()))\n")) sink_fail = true; QString error_messages; if (lib_load_fail) { error_messages.append (i18n ("

\t- The 'rkward' R-library either could not be loaded at all, or not in the correct version. This may lead to all sorts of errors, from single missing features to complete failure to function. The most likely cause is that the last installation did not place all files in the correct place. However, in some cases, left-overs from a previous installation that was not cleanly removed may be the cause.

\

You should quit RKWard, now, and fix your installation. For help with that, see http://rkward.kde.org/compiling.

\n")); } if (sink_fail) { error_messages.append (i18n ("

\t-There was a problem setting up the communication with R. Most likely this indicates a broken installation.

\

You should quit RKWard, now, and fix your installation. For help with that, see http://rkward.kde.org/compiling.

\n")); } RBackendRequest req (true, RBackendRequest::Started); req.params["message"] = QVariant (error_messages); // blocks until RKWard is set to go (esp, it has displayed startup error messages, etc.) // in fact, a number of sub-commands are run while handling this request! handleRequest (&req); commandFinished (); // the fake startup command repl_status.eval_depth--; } void RKRBackend::checkObjectUpdatesNeeded (bool check_list) { RK_TRACE (RBACKEND); if (killed) return; /* NOTE: We're keeping separate lists of the items on the search path, and the toplevel symbols in .GlobalEnv here. This info is also present in RObjectList (and it's children). However: a) in a less convenient form, b) in the other thread. To avoid locking, and other complexity, keeping separate lists seems an ok solution. Keep in mind that only the names of only the toplevel objects are kept, here, so the memory overhead should be minimal */ bool search_update_needed = false; bool globalenv_update_needed = false; if (check_list) { // TODO: avoid parsing this over and over again RK_DEBUG (RBACKEND, DL_TRACE, "checkObjectUpdatesNeeded: getting search list"); RCommandProxy *dummy = runDirectCommand ("search ()\n", RCommand::GetStringVector); if (dummy->stringVector () != toplevel_env_names) search_update_needed = true; if (search_update_needed) toplevel_env_names = dummy->stringVector (); delete dummy; // TODO: avoid parsing this over and over again RK_DEBUG (RBACKEND, DL_TRACE, "checkObjectUpdatesNeeded: getting globalenv symbols"); dummy = runDirectCommand ("ls (globalenv (), all.names=TRUE)\n", RCommand::GetStringVector); QStringList new_globalenv_toplevel_names = dummy->stringVector (); if (new_globalenv_toplevel_names.count () != global_env_toplevel_names.count ()) { globalenv_update_needed = true; } else { for (int i = 0; i < new_globalenv_toplevel_names.count (); ++i) { // order is not important in the symbol list if (!global_env_toplevel_names.contains (new_globalenv_toplevel_names[i])) { globalenv_update_needed = true; break; } } } if (globalenv_update_needed) global_env_toplevel_names = new_globalenv_toplevel_names; delete dummy; if (search_update_needed) { // this includes an update of the globalenv, even if not needed QStringList call ("syncenvs"); call.append (QString::number (toplevel_env_names.size ())); call.append (toplevel_env_names); dummy = runDirectCommand ("loadedNamespaces ()\n", RCommand::GetStringVector); call.append (dummy->stringVector ()); delete dummy; handleHistoricalSubstackRequest (call); } if (globalenv_update_needed) { QStringList call = global_env_toplevel_names; call.prepend ("syncglobal"); // should be faster than the reverse handleHistoricalSubstackRequest (call); } } if (search_update_needed || globalenv_update_needed) { RK_DEBUG (RBACKEND, DL_TRACE, "checkObjectUpdatesNeeded: updating watches"); runDirectCommand (".rk.watch.globalenv ()\n"); } if (!changed_symbol_names.isEmpty ()) { QStringList call = changed_symbol_names; call.prepend (QString ("sync")); // should be faster than reverse handleHistoricalSubstackRequest (call); changed_symbol_names.clear (); } } bool RKRBackend::doMSleep (int msecs) { // do not trace! if (isKilled ()) return false; RKRBackendProtocolBackend::msleep (msecs); return true; } diff --git a/rkward/rbackend/rkrbackendprotocol_backend.cpp b/rkward/rbackend/rkrbackendprotocol_backend.cpp index df549c12..eaa285ab 100644 --- a/rkward/rbackend/rkrbackendprotocol_backend.cpp +++ b/rkward/rbackend/rkrbackendprotocol_backend.cpp @@ -1,173 +1,177 @@ /*************************************************************************** rkrbackendprotocol - description ------------------- begin : Thu Nov 04 2010 copyright : (C) 2010, 2013 by Thomas Friedrichsmeier email : thomas.friedrichsmeier@kdemail.net ***************************************************************************/ /*************************************************************************** * * * This program is free software; you can redistribute it and/or modify * * it under the terms of the GNU General Public License as published by * * the Free Software Foundation; either version 2 of the License, or * * (at your option) any later version. * * * ***************************************************************************/ #include "rkrbackendprotocol_backend.h" #include "rkrbackend.h" #include "../debug.h" #include #include - #include #include + #include "rktransmitter.h" #include #include "rkbackendtransmitter.h" #include // mis-used as a random-string generator #include #include #ifdef Q_OS_MACOS #include #endif void RK_setupGettext (const char*); QMutex RK_Debug_Mutex; void RKDebugMessageOutput (QtMsgType type, const QMessageLogContext &, const QString &msg) { RK_Debug_Mutex.lock (); if (type == QtFatalMsg) { fprintf (stderr, "%s\n", qPrintable (msg)); } RK_Debug::debug_file->write (qPrintable (msg)); RK_Debug::debug_file->write ("\n"); RK_Debug::debug_file->flush (); RK_Debug_Mutex.unlock (); } // NOTE: This function serves no benefit over qDebug() in the backend. But provided for consistency with the frontend. // See the frontend version in main.cpp void RKDebug (int flags, int level, const char *fmt, ...) { Q_UNUSED (flags); Q_UNUSED (level); const int bufsize = 1024*8; char buffer[bufsize]; va_list ap; va_start (ap, fmt); vsnprintf (buffer, bufsize-1, fmt, ap); va_end (ap); RKDebugMessageOutput (QtDebugMsg, QMessageLogContext (), buffer); } int main(int argc, char *argv[]) { #ifdef Q_OS_MACOS CFBundleRef mainBundle = CFBundleGetMainBundle(); if (mainBundle) { // get the application's Info Dictionary. For app bundles this would live in the bundle's Info.plist, // for regular executables it is obtained in another way. CFMutableDictionaryRef infoDict = (CFMutableDictionaryRef) CFBundleGetInfoDictionary(mainBundle); if (infoDict) { // Add or set the "LSUIElement" key with/to value "1". This can simply be a CFString. CFDictionarySetValue(infoDict, CFSTR("LSUIElement"), CFSTR("1")); // That's it. We're now considered as an "agent" by the window server, and thus will have // neither menubar nor presence in the Dock or App Switcher. } } #endif QCoreApplication app (argc, argv); setvbuf (stdout, NULL, _IONBF, 0); setvbuf (stderr, NULL, _IONBF, 0); RK_Debug::RK_Debug_Flags = RBACKEND; if (RK_Debug::setupLogFile (QDir::tempPath () + "/rkward.rbackend")) qInstallMessageHandler (RKDebugMessageOutput); QString servername, rkd_server_name; QString data_dir, locale_dir; QStringList args = app.arguments (); for (int i = 1; i < args.count (); ++i) { if (args[i].startsWith (QLatin1String ("--debug-level"))) { RK_Debug::RK_Debug_Level = args[i].section ('=', 1).toInt (); } else if (args[i].startsWith (QLatin1String ("--server-name"))) { servername = QUrl::fromPercentEncoding (args[i].section ('=', 1).toUtf8 ()); } else if (args[i].startsWith (QLatin1String ("--data-dir"))) { data_dir = QUrl::fromPercentEncoding (args[i].section ('=', 1).toUtf8 ()); } else if (args[i].startsWith (QLatin1String ("--locale-dir"))) { locale_dir = QUrl::fromPercentEncoding (args[i].section ('=', 1).toUtf8 ()); } else if (args[i].startsWith (QLatin1String ("--rkd-server-name"))) { rkd_server_name = QUrl::fromPercentEncoding (args[i].section ('=', 1).toUtf8 ()); } else { printf ("unknown argument %s", qPrintable (args[i])); } } if (servername.isEmpty ()) { printf ("no server to connect to\n"); return 1; } // a simple security token to send to the frontend to make sure that it is really talking to the backend process that it started in the local socket connection. // this token is sent both via stdout and the local socket connection. The frontend simply compares both values. QString token = QUuid::createUuid ().toString (); std::cout << token.toLocal8Bit ().data () << "\n"; std::cout.flush (); RKRBackendTransmitter transmitter (servername, token); RKRBackendProtocolBackend backend (data_dir, rkd_server_name); transmitter.start (); RKRBackend::this_pointer->run (locale_dir); - transmitter.quit (); + RK_DEBUG(RBACKEND, DL_DEBUG, "Main loop finished"); + + QMetaObject::invokeMethod(&transmitter, [&transmitter]() { transmitter.doExit(); }, Qt::QueuedConnection); transmitter.wait (5000); if (!RKRBackend::this_pointer->isKilled ()) RKRBackend::tryToDoEmergencySave (); + QMetaObject::invokeMethod(&app, [&app]() { app.quit(); }, Qt::QueuedConnection); + exit(0); } RKRBackendProtocolBackend* RKRBackendProtocolBackend::_instance = 0; RKRBackendProtocolBackend::RKRBackendProtocolBackend (const QString &storage_dir, const QString &_rkd_server_name) { RK_TRACE (RBACKEND); _instance = this; new RKRBackend (); r_thread = QThread::currentThread (); // R thread == main thread #ifndef Q_OS_WIN r_thread_id = QThread::currentThreadId (); #endif data_dir = storage_dir; rkd_server_name = _rkd_server_name; } RKRBackendProtocolBackend::~RKRBackendProtocolBackend () { RK_TRACE (RBACKEND); } void RKRBackendProtocolBackend::sendRequest (RBackendRequest *_request) { RK_TRACE (RBACKEND); RBackendRequest* request = _request; if (!request->synchronous) { request = _request->duplicate (); // the instance we send to the frontend will remain in there, and be deleted, there _request->done = true; // for aesthetics } RKRBackendEvent* event = new RKRBackendEvent (request); RK_ASSERT (request->type != RBackendRequest::Output); qApp->postEvent (RKRBackendTransmitter::instance (), event); } bool RKRBackendProtocolBackend::inRThread () { return (QThread::currentThread () == instance ()->r_thread); } void RKRBackendProtocolBackend::msleep (int delay) { static_cast (RKRBackendTransmitter::instance ())->publicmsleep (delay); } QString RKRBackendProtocolBackend::backendDebugFile () { return RK_Debug::debug_file->fileName (); } diff --git a/rkward/rbackend/rkrbackendprotocol_frontend.cpp b/rkward/rbackend/rkrbackendprotocol_frontend.cpp index 6f3dd439..b9b73d33 100644 --- a/rkward/rbackend/rkrbackendprotocol_frontend.cpp +++ b/rkward/rbackend/rkrbackendprotocol_frontend.cpp @@ -1,102 +1,102 @@ /*************************************************************************** rkrbackendprotocol - description ------------------- begin : Thu Nov 04 2010 copyright : (C) 2010, 2011 by Thomas Friedrichsmeier email : thomas.friedrichsmeier@kdemail.net ***************************************************************************/ /*************************************************************************** * * * This program is free software; you can redistribute it and/or modify * * it under the terms of the GNU General Public License as published by * * the Free Software Foundation; either version 2 of the License, or * * (at your option) any later version. * * * ***************************************************************************/ #include "rkrbackendprotocol_frontend.h" #include "rkrinterface.h" #include #include "rkfrontendtransmitter.h" #include #include "../debug.h" RKRBackendProtocolFrontend* RKRBackendProtocolFrontend::_instance = 0; RKRBackendProtocolFrontend::RKRBackendProtocolFrontend (RInterface* parent) : QObject (parent) { RK_TRACE (RBACKEND); RK_ASSERT (!_instance); frontend = parent; _instance = this; } RKRBackendProtocolFrontend::~RKRBackendProtocolFrontend () { RK_TRACE (RBACKEND); terminateBackend (); - RKFrontendTransmitter::instance ()->wait (1000); // Wait for thread to catch the backend's exit request, and exit() - RKFrontendTransmitter::instance ()->quit (); // Tell it to quit, otherwise - RKFrontendTransmitter::instance ()->wait (1000); // Wait for thread to quit and clean up. + RKFrontendTransmitter::instance ()->wait(1000); // Wait for thread to catch the backend's exit request, and exit() + RKFrontendTransmitter::instance ()->quit(); // Tell it to quit, otherwise + RKFrontendTransmitter::instance ()->wait(3000); // Wait for thread to quit and clean up. delete RKFrontendTransmitter::instance (); } void RKRBackendProtocolFrontend::setupBackend () { RK_TRACE (RBACKEND); new RKFrontendTransmitter (); } void RKRBackendProtocolFrontend::setRequestCompleted (RBackendRequest *request) { RK_TRACE (RBACKEND); bool sync = request->synchronous; request->completed (); if (!sync) return; RKRBackendEvent* ev = new RKRBackendEvent (request); qApp->postEvent (RKFrontendTransmitter::instance (), ev); QThread::yieldCurrentThread (); } ROutputList RKRBackendProtocolFrontend::flushOutput (bool force) { return static_cast (RKFrontendTransmitter::instance ())->flushOutput (force); } void RKRBackendProtocolFrontend::interruptCommand (int command_id) { RK_TRACE (RBACKEND); RBackendRequest *req = new RBackendRequest (false, RBackendRequest::Interrupt); req->params.insert ("commandid", QVariant (command_id)); qApp->postEvent (RKFrontendTransmitter::instance (), new RKRBackendEvent (req)); } void RKRBackendProtocolFrontend::sendPriorityCommand (RCommandProxy* proxy) { RK_TRACE (RBACKEND); RBackendRequest *req = new RBackendRequest (false, RBackendRequest::PriorityCommand); req->command = proxy; qApp->postEvent (RKFrontendTransmitter::instance (), new RKRBackendEvent (req)); } void RKRBackendProtocolFrontend::terminateBackend () { RK_TRACE (RBACKEND); // Backend process will terminate automatically, when the transmitter dies } void RKRBackendProtocolFrontend::customEvent (QEvent *e) { if (((int) e->type ()) == ((int) RKRBackendEvent::RKWardEvent)) { RKRBackendEvent *ev = static_cast (e); frontend->handleRequest (ev->data ()); } else { RK_ASSERT (false); return; } }