* Attempt to fix test workflows execution
* Remove apt update
* trying to fix timezone
* Trying to fix build issues
* Reduced concurrency level
* Improve error message and further decrease concurrency level
* Further reduce concurrency level
* Removed concurrency
* Removing update credentials as it's not working currently
* Also removing export
* Fixed versions for typeorm and sqlite
* Re add concurrency
* Adjustig sqlite3 and typeorm versions
* Fixing only sqlite version
* Fixing only typeorm version
* Reverting unnecessary changes
* Improved the error object returned by axios to make it more compatible with request
* Fixed multipart/form-data payload creation
* 🐛 Remove issue with circular references
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* ⚡ introduce versioned nodes
* Export versioned nodes for separate process run
* Add bse node for versioned nodes
* fix node name for versioned nodes
* extend node from nodeVersionedType
* improve nodes base and flow to FE
* revert lib es2019 to es2017
* include version in key to prevent duplicate key
* handle type versions on FE
* clean up
* cleanup nodes base
* add type versions in getNodeParameterOptions
* cleanup
* code review
* code review + add default version to node type description
* remove node default types from store
* 💄 cleanups
* Draft for migrated Mattermost node
* First version of Mattermost node versioned according to node standards
* Correcting deactivate operations name to match currently used one
* ✨ Create utility types
* ⚡ Simplify Mattermost types
* ⚡ Rename exports for consistency
* ⚡ Type channel properties
* ⚡ Type message properties
* ⚡ Type reaction properties
* ⚡ Type user properties
* ⚡ Add type import to router
* 🐛 Add missing key
* 🔨 Adjust typo in operation name
* 🔨 Inline exports for channel properties
* 🔨 Inline exports for message properties
* 🔨 Inline exports for reaction properties
* 🔨 Inline exports for user properties
* 🔨 Inline exports for load options
* 👕 Fix lint issue
* 🔨 Inline export for description
* 🔨 Rename descriptions for clarity
* 🔨 Refactor imports/exports for methods
* 🔨 Refactor latest version retrieval
* 🔥 Remove unneeded else clause
When the string literal union is exhausted, the resource key becomes never, so TS disallows wrong key usage.
* ✨ Add overloads to getNodeParameter
* ⚡ Improve overload
* 🔥 Remove superfluous INodeVersions type
* 🔨 Relocate pre-existing interface
* 🔥 Remove JSDoc arg descriptions
* ⚡ Minor reformatting in transport file
* ⚡ Fix API call function type
* Created first draft for Axios requests
* Working version of mattermost node with Axios
* Work in progress for replacing request library
* Improvements to request translations
* Fixed sending files via multipart / form-data
* Fixing translation from request to axios and loading node parameter options
* Improved typing for new http helper
* Added ignore any for specific lines for linting
* Fixed follow redirects changes on http request node and manual execution of previously existing workflow with older node versions
* Adding default headers according to body on httpRequest helper
* Spec error handling and fixed workflows with older node versions
* Showcase how to export errors in a standard format
* Merging master
* Refactored mattermost node to keep files in a uniform structure. Also fix bugs with merges
* Reverting changes to http request node
* Changed nullish comparison and removed repeated code from nodes
* Renamed queryString back to qs and simplified node output
* Simplified some comparisons
* Changed header names to be uc first
* Added default user agent to requests and patch http method support
* Fixed indentation, remove unnecessary file and console log
* Fixed mattermost node name
* Fixed lint issues
* Further fix linting issues
* Further fix lint issues
* Fixed http request helper's return type
Co-authored-by: ahsan-virani <ahsan.virani@gmail.com>
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
* Changes to types so that credentials can be always loaded from DB
This first commit changes all return types from the execute functions
and calls to get credentials to be async so we can use await.
This is a first step as previously credentials were loaded in memory and
always available. We will now be loading them from the DB which requires
turning the whole call chain async.
* Fix updated files
* Removed unnecessary credential loading to improve performance
* Fix typo
* ⚡ Fix issue
* Updated new nodes to load credentials async
* ⚡ Remove not needed comment
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Added try catch blocks to avoid endlessly running workflows
* Added handling for subworkflows
* ⚡ Fix one cause of "unkown" status of worklows with "main" mode
* ⚡ Fix one cause of "unkown" status of worklows with "own" mode
* ⚡ Fix one cause of "unkown" status of worklows with "queue" mode
* Saving database recovery
* 🐛 Fix issue that errors did not get saved correctly and also not
displayed
* ⚡ Save workflow timeout correctly as error
* Adding error capture to queued jobs
* ⚡ Mark canceled executions as not finished consistently across all
modes
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Performance improvements for executions count on Postgres
As reported by a community member https://community.n8n.io/t/stress-load-testing/4846/5
and https://github.com/n8n-io/n8n/issues/1578, when using postgres with
a big volume of executions, the executions list's performance degrades.
This PR is aimed at Postgres specifically by querying postgres' stats
collector instead of running a full table scan, providing a good
estimate.
More can be read here: https://www.citusdata.com/blog/2016/10/12/count-performance/
* Removed order of magnitude so we display closer numbers
* Making count based on statistics only when not applying filters
* ⚡ Minor styling improvements
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Added flag to simplify output to execute command and created executeAll
Also created a command that lists workflows so it can be used by
other applications that wish to interact with n8n via CLI.
* Added compare funcionality that helps us identify possible breaking changes
* ⚡ Add JSON flag to output execution result in json format
* Add execution time to execution result
* Add --output flag to save JSON formated results
* Fix typos
* Adding usage examples and warning message
* Changing command to always output JSON information
* ⚡ Add concurrency to executeAll command
* ✨ Add IDs filter to executeAll command
* ✏️ Fix typos
* Fix lint issues
* Improvements to execute all
* Added colors and beautified execute all command output
* Improving premature termination of execute all command
* Refactored output
* Added detection for warnings instead of errors for a few cases
* Fixed linting
* Improved process finishing
* Added encryption key init, removed messages from non-debug run and created sample github actions file
* Updated test command to use correct encryption key
* Correcting paths
* Changed command name and changed concurrency to be slot based
* Added the retry functionality
* Improved typing and fixed executions filtering
* Updated test description
* Added concurrency
* Added skip list and concurrency
* Fixed termination behavior, removed unnecessary code and added main mode to executions
* Added special edge cases handling
* Added safe debug flag for non tty terminals
* Removed comparison and retries for testing
* Changed the way we run commands to add more debug info
* Test adding concurrency
* Testing with regular testing procedure without comparison
* Adding compare flag
* Fixing short output command
* Upgraded concurrency to 16, fixed short output and covered nodes count
* Fixed short output condition
* Correcting short output information
* Correcting the output for short output
* Added a string to possible warnings list
* Added example usages with more arguments
* Improvements to code made by Ivan
* Removed colorize output setting (always use colors), moved execution timeout to become static and removed node list from executions
* Moving types to a separate file on execute batch command
* Changed interfaces file extension to prevent oclif warnings
* Updated workflow pdf files
* ⚡ Change rule when tests run + minor formatting fixes
Co-authored-by: dali <servfrdali@yahoo.fr>
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Added the option to flatten output so we can more easily work with arrays
* ⚡ Change parameter name
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Migration for MySQL to certify that we're using the correct collation
* Update migration to check for collation existence before running
* ⚡ Fix format
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Added logging to n8n
This commit adds logging to n8n using the Winston library.
For now, this commit only allows logging to console (default behavior)
or file (need to pass in config via environment variables).
Other logging methods can be further implemented using hooks. These were
skipped for now as it would require adding more dependencies.
Logging level is notice by default, meaning no additional messages would
be displayed at the moment. Logging level can be set to info or debug as
well to enrich the generated logs.
The ILogger interface was added to the workflow project as it would make
it available for all other projects but the implementation was done on
the cli project.
* Lint fixes and logging level naming. Also fixed the way we use the logger as it was not working previously
* Improvements to logging framework
Using appropriate single quotes
Improving the way the logger is declared
* Improved naming for Log Types
* Removed logger global variable, replacing it by a proxy
* Add logging to CLI commands
* Remove unused GenericHelpers
* Changed back some messages to console instead of logger and added npm
shortcuts for worker and webhook
* Fix typos
* Adding basic file rotation to logs as suggested by @mutdmour
* Fixed linting issues
* Correcting comment to correctly reflect space usage
* Added settings for log files rotation
* Correcting config type from String to Number
* Changed default file settings to number
To reflect previous changes to the type
* Changed the way log messages are added to be called statically. Also minor naming improvements
* Applying latest corrections sent by @ivov
* ⚡ Some logging improvements
* Saving logs to a folder inside n8n home instead of root
* Fixed broken tests and linting
* Changed some log messages to improve formatting
* Adding quotes to names on log messages
* Added execution and session IDs to logs. Also removed unnecessary line breaks
* ⚡ Added file caller to log messages (#1657)
This is done using callsites library which already existed
in the project as another library's dependency. So in fact
it does not add any new dependency.
* Adding logs to help debug Salesforce node
* ⚡ Add function name to logs and add more logs
* ⚡ Improve some error messages
* ⚡ Improve some more log messages
* ⚡ Rename logging env variables to match others
Co-authored-by: dali <servfrdali@yahoo.fr>
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Adding support to ParameterizedQuery on Postgres Node
* Created another parameter to toggle on replacement so it's clear to users what is happening.
* Fixed lint issues
* ⚡ Formatting
* Improvements to questDB node so it is more consistent
* Fixed lint issues
* Fixed typing issue
* ⚡ Apply suggestions BHesseldieck
Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com>
* Standardized output for postgres and postgres-like nodes
This changes the behavior of CrateDB, Postgres, QuestDB and TimescaleDB
nodes.
The Execute Query operation used to execute multiple queries but return
the result from only one of the queries.
This change causes the node output to containt results from all queries
that ran, making the behavior more consistent across all n8n.
* Fixing lint issues
* ⚡ Minor improvements
* ⚡ Fix breaking changes files
Co-authored-by: Gustavo Arjones <gustavo.arjones@ank.app>
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
Co-authored-by: Jan <janober@users.noreply.github.com>
Co-authored-by: Ben Hesseldieck <1849459+BHesseldieck@users.noreply.github.com>
* add multi return
* add independently and transaction to query
* pgInsert normal and transaction
* independently for pgInsert
* normal, transaction and independently for pgUpdate
* cleanup
* implement it in other nodes
* multiple fixes
* add optional returning support
* clean up Postgres functions
* fix other postgres based dbs
* Added option to run queries as a transaction to Postgres
This commit allows users to configure Postgres, CrateDB, TimescaleDB and
QuestDB to run queries independently or as transactions as well as the
previous mode which is to execute multiple queries at once.
Previous behavior remains untouched so we only added new options.
* Standardize behavior across nodes that use postgres protocol
Also fixed unit tests.
* Added breaking change notice
* Added more information to breaking changes
* ⚡ Styling fixes
Co-authored-by: lublak <lublak.de@gmail.com>
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Implemented timeout for workers and corrected timeout for subworkflows
* Fixed issue with timeouts when running on separate processes
* Standardized timeouts across all n8n
Now the maxTimeout setting takes effect whenever a timeout is set
to any workflows.
This causes local timeouts (either set on a per-workflow basis or
via global settings) to be capped by the maximum timeout. This
behavior already existed but was not applied to all places.
Also changed the way n8n enforces timeouts for subworkflows, making
it work always.
In effect, with this change, if you have one workflow that calls others
only the main workflow's timeout is taken into consideration, limiting
the maximum time that other workflows combined can run.
* ⚡ Fix timeout problem in "own" mode
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Added a global timeout setting for all http requests except for http
node.
The http node explicitly sets a timeout that overrides the default
value.
This is to prevent behaviors when users were expecting that their quests
could take a very long time to execute.
* Removed unnecessary typing from code
* 538 add support for casting types in postgres
* Add typing to postgres insert nodes and removed unnecessary field from crate db
* Added placeholder and description for types in postgres
* Adding tests to insert and changes suggested by Ben
* ⚡ Minor improvement
Co-authored-by: mutdmour <mutdmour@gmail.com>
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* ✨ n8n start trigger node
* first declaration of WorkflowActivationMode
* implement first WorkflowActivationMode: 'init', 'create', 'update', 'activate'
* fix Server missing id
* add activation infos to triggers
* remove WorkflowActivationMode from webhook execution function
* add some missing activation and add manual activation
* clean up and fix some code
* fix UnhandledPromiseRejectionWarning: Error: Overwrite NodeExecuteFunctions.getExecuteTriggerFunctions.emit function!
* fix spaces
* use a better name for the node
* fix ident in package.json
* Contributions to lublak's PR #1287
* Fixed linting issues and change the way parameters are displayed
* ⚡ Fix name and minor improvements
Co-authored-by: lublak <lublak.de@gmail.com>
Co-authored-by: lublak <44057030+lublak@users.noreply.github.com>
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Created the node for DeepL translator tool
Still missing a few additional fields but functionality is ok for most cases.
* Added optional fields for Deepl and separated description to a separate file
* Fixed linting issue
* 🎨 Replace PNG with SVG icon
* 💄 Adjust style to codebase conventions
* 🔨 Refactor types
* ⚡ Simplify error handling
* ⚡ Add always open edit window for text field
* ✏️ Edit descriptions in text operations
* ⚡ Fix source language for English
EN-GB and EN-US are not supported as source languages, but EN is.
* 💄 Apply cosmetic changes
* ⚡ Small improvement
* ⚡ Remove not needed Authentication selection
Co-authored-by: Iván Ovejero <ivov.src@gmail.com>
Co-authored-by: ricardo <ricardoespinoza105@gmail.com>
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Removing unnecessary data from process communication
* Wrapping progress save in a try ... catch block to prevent crashes
* Migration to fix mysql problems with big data on executions
* ⚡ Minor formatting fixes
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Fix issue with auto refresh on execution list
When auto refresh is on, we used to get executions from backend using
the firstId field to filter recent executions.
This is a problem when you have executions that do not finish in order,
leaving gaps behind. This PR fixes this problem by refreshing the latest
30 executions and correctly adding them to the list.
* Fixed an issues with auto refresh on executions ExecutionsList
Fixed two bugs, one in frontend which was ignoring the first returned
row from the backend and an issue with backend that was not using
the overriden version of `executeWorkflow` function for sub sub
workflows.
* Fixed the display of manual executions when running with queues and improved display of subworkflows
* Changing workflow ids array from variable to constant
* Added unknown status to workflow execution and changed its color to orange
When running n8n in queue mode, the query to search for currently
running workflows would fail on Postgres but work fine with SQLite and
MySQL / MariaDB. This fix makes it work fine for all RDBMS.
* Unify execution ID across executions
* Fix indentation and improved comments
* WIP: saving data after each node execution
* Added on/off to save data after each step, saving initial data and retries working
* Fixing lint issues
* Fixing more lint issues
* ✨ Add bull to execute workflows
* 👕 Fix lint issue
* ⚡ Add graceful shutdown to worker
* ⚡ Add loading staticData to worker
* 👕 Fix lint issue
* ⚡ Fix import
* Changed tables metadata to add nullable to stoppedAt
* Reload database on migration run
* Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration
* Added checks to Redis and exiting process if connection is unavailable
* Fixing error with new installations
* Fix issue with data not being sent back to browser on manual executions with defined destination
* Merging bull and unify execution id branch fixes
* Main process will now get execution success from database instead of redis
* Omit execution duration if execution did not stop
* Fix issue with execution list displaying inconsistant information information while a workflow is running
* Remove unused hooks to clarify for developers that these wont run in queue mode
* Added active pooling to help recover from Redis crashes
* Lint issues
* Changing default polling interval to 60 seconds
* Removed unnecessary attributes from bull job
* Added webhooks service and setting to disable webhooks from main process
* Fixed executions list when running with queues. Now we get the list of actively running workflows from bull.
* Add option to disable deregistration of webhooks on shutdown
* Rename WEBHOOK_TUNNEL_URL to WEBHOOK_URL keeping backwards compat.
* Added auto refresh to executions list
* Improvements to workflow stop process when running with queues
* Refactor queue system to use a singleton and avoid code duplication
* Improve comments and remove unnecessary commits
* Remove console.log from vue file
* Blocking webhook process to run without queues
* Handling execution stop graciously when possible
* Removing initialization of all workflows from webhook process
* Refactoring code to remove code duplication for job stop
* Improved execution list to be more fluid and less intrusive
* Fixing workflow name for current executions when auto updating
* ⚡ Right align autorefresh checkbox
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Unify execution ID across executions
* Fix indentation and improved comments
* WIP: saving data after each node execution
* Added on/off to save data after each step, saving initial data and retries working
* Fixing lint issues
* Fixing more lint issues
* ✨ Add bull to execute workflows
* 👕 Fix lint issue
* ⚡ Add graceful shutdown to worker
* ⚡ Add loading staticData to worker
* 👕 Fix lint issue
* ⚡ Fix import
* Changed tables metadata to add nullable to stoppedAt
* Reload database on migration run
* Fixed reloading database schema for sqlite by reconnecting and fixing postgres migration
* Added checks to Redis and exiting process if connection is unavailable
* Fixing error with new installations
* Fix issue with data not being sent back to browser on manual executions with defined destination
* Merging bull and unify execution id branch fixes
* Main process will now get execution success from database instead of redis
* Omit execution duration if execution did not stop
* Fix issue with execution list displaying inconsistant information information while a workflow is running
* Remove unused hooks to clarify for developers that these wont run in queue mode
* Added active pooling to help recover from Redis crashes
* Lint issues
* Changing default polling interval to 60 seconds
* Removed unnecessary attributes from bull job
* ⚡ Improved output on worker job start
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Added options to import and export workflows and credentials
* Fixed linting issues and descriptions
* Linting issues and allowing import/export operations to output multiple files
* ⚡ Minor changes to CLI export/import
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
* Allowing Gmail node to send messages formatted as HTML
* Improving message field description
* Fixing lint issues
* Adding missing trailing comma
* ⚡ Small improvements to GMail-Node
Co-authored-by: Jan Oberhauser <jan.oberhauser@gmail.com>
Yahoo mail is unable to provide a response in under 3 seconds. Increasing
this limit to 10 seconds seems to solve the timeout issue.
I was unable to properly login to Yahoo but at least the timeout error
is not happening anymore.
Setting the node to work with Gmail worked fine although it is a
somewhat painful process as Gmail avoids the so called non secure apps.
Reported by community user huuich (https://community.n8n.io/t/gmail-integration-partly-created/693/12?u=jan)
An attempt to fix the problem using basic JS functions (escape / unescape
/ encoding / decoding) did not work, so I decided to use MailComposer
as suggested by Jan on Jira Issue PROD-575
* ⚡ Changed rest api endpoint to omit node properties by default.
⚡ Created another endpoint to return all node information based on a list of names
* refactor: changed endpoint to POST instead of GET
* Changed properties to be optional for node description
* Removed eager loading for node properties.
All nodes will have their properties fetched when used in a workflow.
Works when opening an already saved workflow, creating a new one from
scratch or pasting JSON / URLs.
* Removing unnecessary dependency
Omar Ajoue