{"id":3592,"date":"2017-09-27T10:24:46","date_gmt":"2017-09-27T09:24:46","guid":{"rendered":"https:\/\/www.gamlor.info\/wordpress\/?p=3592"},"modified":"2021-03-11T09:42:56","modified_gmt":"2021-03-11T08:42:56","slug":"adding-timestamp-to-logs","status":"publish","type":"post","link":"https:\/\/www.gamlor.info\/wordpress\/2017\/09\/adding-timestamp-to-logs\/","title":{"rendered":"Adding Timestamp to Logs"},"content":{"rendered":"

How many small script or process do you have which just log to stdout\/stderr? A few, dozens?
\nTheir running as cronjobs or service?
\nStashing the logs somewhere, like:<\/p>\n

\r\n.\/produces-some-log.sh 2>&1 > log-of-my-script.log\r\n<\/pre>\n
Do you have time stamps on those logs?
\nIf not, it is trivial to add. You just have to pipe the log through a time stamp tool.<\/p>\n
\"\"<\/a>
Timestamp your lines<\/p><\/div>\n
Linux: ts<\/h2>\n
On most Linux distros there the `ts` tool. If it is not installed, it is usually in the ‘moreutils’ package. So get it via `apt-get`, `yum`, `pacman` or whatever you`re distro uses.<\/p>\n
Then, pipe the output through `ts`:<\/p>\n
\r\n[gamlor@gamlor-manjaro log-util]$ .\/produces-some-log.sh 2>&1                                          \r\nWe are making progress. We are at step 1\/5         \r\nAnd some other info about the system:  13:39:24 up 2:10, 2 users, load average: 1.44, 1.74, 1.76       \r\nWe are making progress. We are at step 2\/5         \r\nAnd some other info about the system:  13:39:25 up 2:10, 2 users, load average: 1.44, 1.74, 1.76       \r\nWe are making progress. We are at step 3\/5         \r\n....\r\n<\/pre>\n
Will become:<\/p>\n
\r\n[gamlor@gamlor-manjaro log-util]$ .\/produces-some-log.sh 2>&1 | ts                                     \r\nSep 25 13:39:30 We are making progress. We are at step 1\/5                                             \r\nSep 25 13:39:30 And some other info about the system:  13:39:30 up 2:10, 2 users, load average: 1.65, 1.78, 1.77                                                                                               \r\nSep 25 13:39:31 We are making progress. We are at step 2\/5                                             \r\nSep 25 13:39:31 And some other info about the system:  13:39:31 up 2:10, 2 users, load average: 1.65, 1.78, 1.77                                                                                               \r\nSep 25 13:39:31 We are making progress. We are at step 3\/5                                             \r\n...          \r\n<\/pre>\n
You can specify a date format string to have more clear format:<\/p>\n
\r\n[gamlor@gamlor-manjaro log-util]$ .\/produces-some-log.sh 2>&1 | ts '%Y-%m-%dT%T%z'\r\n2017-09-25T22:47:25+0900 We are making progress. We are at step 1\/5\r\n2017-09-25T22:47:25+0900 And some other info about the system:  22:47:25 up 2:18, 2 users, load average: 1.25, 1.29, 1.53\r\n2017-09-25T22:47:26+0900 We are making progress. We are at step 2\/5\r\n2017-09-25T22:47:26+0900 And some other info about the system:  22:47:26 up 2:18, 2 users, load average: 1.25, 1.29, 1.53\r\n...\r\n<\/pre>\n
Last, I recommend using the UTC time zone, to avoid confusion. Expecially when your servers are all over the world. Or you need to cross reference time stamps with other logs.
\nIf your server isn`t set to UTC, you can set the `TZ` enviroment variable:<\/p>\n
\r\n[gamlor@gamlor-manjaro log-util]$ export TZ=UTC\r\n[gamlor@gamlor-manjaro log-util]$ .\/produces-some-log.sh 2>&1 | ts '%Y-%m-%dT%T%z'\r\n2017-09-25T13:49:54+0000 We are making progress. We are at step 1\/5\r\n2017-09-25T13:49:54+0000 And some other info about the system:  13:49:54 up 2:21, 2 users, load average: 0.66, 1.13, 1.44\r\n2017-09-25T13:49:55+0000 We are making progress. We are at step 2\/5\r\n...\r\n<\/pre>\n
For more details, check the man pages<\/a>.<\/p>\n
For all Unix-like systems, AWK to the rescue!<\/h2>\n
BSD, illumos or some other Unix like system? Maybe there no `ts` util around? Good old awk has got you covered. On each line, print the date, then the rest of the line.<\/p>\n
This method is more portable. AWK is installed most systems. So, if you plan do distribute your script, this is the way to go.<\/p>\n
\r\n[gamlor@gamlor-manjaro log-util]$ .\/produces-some-log.sh 2>&1 | awk '{ print strftime(\"%Y-%m-%dT%T%z\"), $0;  }'                                                                                                     \r\n2017-09-25T22:57:54+0900 We are making progress. We are at step 1\/5                                    \r\n2017-09-25T22:57:54+0900 And some other info about the system:  22:57:54 up 2:29, 2 users, load average: 1.29, 1.23, 1.39                                                                                      \r\n2017-09-25T22:57:55+0900 We are making progress. We are at step 2\/5                                    \r\n...\r\n<\/pre>\n
Again, I recommend using UTC. Your server probably already uses UTC anyway:<\/p>\n
\r\n[gamlor@gamlor-manjaro log-util]$ export TZ=UTC    \r\n[gamlor@gamlor-manjaro log-util]$ .\/produces-some-log.sh 2>&1 | awk '{ print strftime(\"%Y-%m-%dT%T%z\"), $0;  }'                                                                                                     \r\n2017-09-25T13:59:23+0000 We are making progress. We are at step 1\/5                                    \r\n2017-09-25T13:59:23+0000 And some other info about the system:  13:59:23 up 2:30, 2 users, load average: 1.09, 1.15, 1.35                                                                                      \r\n2017-09-25T13:59:24+0000 We are making progress. We are at step 2\/5                                    \r\n...                                  \r\n<\/pre>\n
That`s it =).
\nThanks for the Stackoverflow answers<\/a>, which include both solutions.<\/p>\n","protected":false},"excerpt":{"rendered":"
How many small script or process do you have which just log to stdout\/stderr? A few, dozens? Their running as cronjobs or service? Stashing the logs somewhere, like: .\/produces-some-log.sh 2>&1… <\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"closed","ping_status":"closed","sticky":false,"template":"","format":"standard","meta":{"footnotes":"","_links_to":"","_links_to_target":""},"categories":[268,187,19],"tags":[312,309],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/posts\/3592"}],"collection":[{"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/comments?post=3592"}],"version-history":[{"count":9,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/posts\/3592\/revisions"}],"predecessor-version":[{"id":3602,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/posts\/3592\/revisions\/3602"}],"wp:attachment":[{"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/media?parent=3592"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/categories?post=3592"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/www.gamlor.info\/wordpress\/wp-json\/wp\/v2\/tags?post=3592"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}