Org Projects with Gantt Charts
Table of Contents
Org-Mode is a great way to store TODO items, store notes, and keep track of
projects. However, possibly due to ignorance of what is available, getting a
concrete high-level view of a project is difficult. There are numerous
different visualizations that can be created to see the state of a project.
Each answers a specific question, none are entirely complete. Today, I have
worked out a mechanism to generate Gantt Charts using dynamic blocks for a
specific project tree. In followup posts, I may work out some other
visualizations.
Agenda views can help gather and provide filtered projections of the TODO lists
that comprise various projects. But getting a holistic view of the progress is
not entirely captured by progress cookies and agenda views. For example, a
progress cookie cannot predict a project's completion because it lacks
information about size of each task and estimated complexity of each task. A
project could be 98% complete, but the last 2% of tasks consist of 80% of the
work necessary to complete. Arguably, this is a failure in the project
breakdown, but the point remains, some amount of tasks left over as a whole do
not communicate the holistic progress and prediction towards completion.
Background
Gantt charts model project schedules as a sort of bar chart. Dependencies and links can be shown to draw the critical path of the project and the timeline of the various tasks that makeup a project. These project schedules are more common in waterfall project management techniques or resource scheduling, therefore, they are likely less popular than other visualizations. However, the critical path projection is still an important view of any project management philosophy. Furthermore, the complexity and cost of generating Gantt charts has made them unlikely visualizations even for smaller task breakdowns. Another important aspect that Gantt charts model is task duration as either an abstract unit, such as "story points", or as calendar dates with some chosen granularity of time units such as days, months, or years.
Prior Art
There are several existing projects which attempt to generate Gantt charts from Org TODO entries. Specifically, Org mode supports exporting directly to TaskJuggler, there is even a tutorial on the Org mode website. There are also third-party packages such as elgantt and Org-Gantt.
Rationale
Given that there are existing solutions to generating Gantt charts from Org mode entries, why create another one? The other solutions require either the Org entries change or metadata added such that the chart generator of choice has the necessary information. Instead, it is better to leverage the existing information already present entries to generate the charts.
Implementation
Currently, the implementation is not packaged, but that may change at a future date.
The basics of the implementation involves Emacs Lisp, Org mode, \LaTeX, TikZ,
and pgfGantt. Using an existing (sub)tree of tasks (Org TODO entries), we
create a dynamic block that processes the current tree and generates the
necessary \ganttbars for generating a Gantt Chart using pgfGantt. Then, the
document or subtree can be exported to PDF using \LaTeX to create Gantt chart
projection.
We define a special function, org-dblock-write:gantt which is used to generate
the necessary TikZ/pgfGantt commands to draw the chart.
(require 'org) (require 'seq) (defun org-dblock-write:gantt (params) "Create pgf Gantt Chart from subtree." (defun org-parse-date (date-string) (cond ((null date-string) nil) (t (seconds-to-time (org-matcher-time date-string))))) (defun org-duration->minutes (duration-string) "Parse DURATION-STRING into numerical minutes." (cond ((org-duration-p duration-string) (org-duration-to-minutes duration-string)) (t 0))) (defun org--alist-entry (&optional filter-level) (let* ((props (org-entry-properties)) (level (cl-first (org-heading-components))) (entry-title (cdr (assoc "ITEM" props))) (entry-id (cdr (assoc "ID" props))) (effort-string (cdr (assoc "EFFORT" props))) (clock-minutes (org-clock-sum-current-item)) (scheduled (cdr (assoc "SCHEDULED" props))) (deadline (cdr (assoc "DEADLINE" props))) (effort-minutes (org-duration->minutes effort-string)) (status (cl-third (org-heading-components))) (done (seq-contains-p org-done-keywords status)) (progress (cond (done 100.0) ((not (equal effort-minutes 0)) (* (/ clock-minutes effort-minutes) 100)) (t nil)))) `((LEVEL . ,level) (TYPE . ,(cond ((< level filter-level) 'GROUP) (t 'BAR))) (TITLE . ,entry-title) (ID . ,entry-id) (EFFORT . ,effort-minutes) (CLOCKED . ,clock-minutes) (SCHEDULED . ,scheduled) (DEADLINE . ,deadline) (PROGRESS . ,progress) (DONE . ,done)))) (defun format-entry (entry) (defun format-progress (value) (if (null value) 0 value)) (defun format-title (title) (let* ((replaced-title (string-replace "%" "\\%" title)) (truncate-to (min (length replaced-title) 15))) (substring replaced-title 0 truncate-to))) (defun format-date (date) (format-time-string "%Y-%m-%d" date)) (let-alist entry (cond ((equal 'GROUP .TYPE) (format "\\ganttgroup[progress=today]{%s}{%s}{%s} \\\\\n" (format-title .TITLE) (format-date .SCHEDULED) (format-date .DEADLINE))) (t (format "\\ganttbar[progress=%00.0f]{%s}{%s}{%s} \\\\\n" (format-progress .PROGRESS) (format-title .TITLE) (format-date .SCHEDULED) (format-date .DEADLINE)))))) (let* ((start (org-parse-date (plist-get params :tstart))) (end (org-parse-date (plist-get params :tend))) (today (org-parse-date "<today>")) (current-level (+ 1 (cl-first (org-heading-components)))) (level (or (plist-get params :level) current-level)) (tunit (or (plist-get params :tunit) "month")) (entries (seq-filter (lambda (entry) (let-alist entry (<= .LEVEL level))) (org-map-entries (lambda () (org--alist-entry level)) t 'tree)))) (insert (format "#+begin_src latex \\begin{ganttchart}[%% expand chart=\\textwidth, vgrid, hgrid, time slot format=isodate, bar height=0.6, bar label font=\\scriptsize, bar/.append style={fill=green!50}, bar incomplete/.append style={fill=red!50}, group/.append style={fill=blue!50}, group incomplete/.append style={fill=brown!50}, group left shift=0, group right shift=0, group top shift=.6, group height=.3, group peaks height=.2, today=%s, time slot unit=%s]{%s}{%s}\n" (format-time-string "%Y-%m-%d" today) tunit (format-time-string "%Y-%m-%d" start) (format-time-string "%Y-%m-%d" end))) (insert (format "\\gantttitlecalendar{year, month%s} \\\\\n" (if (equal tunit "day") ", day" ""))) (let ((previous-end (decode-time nil (current-time-zone) t))) (cl-map nil (lambda (entry) (let* ((id (cdr (assoc 'ID entry))) (level (cdr (assoc 'LEVEL entry))) (type (cdr (assoc 'TYPE entry))) (title (cdr (assoc 'TITLE entry))) (clocked (cdr (assoc 'CLOCKED entry))) (done (cdr (assoc 'DONE entry))) (effort (cdr (assoc 'EFFORT entry))) (scheduled (or (org-parse-date (cdr (assoc 'SCHEDULED entry))) (encode-time previous-end))) (deadline (or (org-parse-date (cdr (assoc 'DEADLINE entry))) (encode-time (decoded-time-add (decode-time scheduled) (make-decoded-time :minute effort))))) (progress (cdr (assoc 'PROGRESS entry))) (task `((ID . ,id) (PROGRESS . ,progress) (TYPE . ,type) (LEVEL . ,level) (TITLE . ,title) (EFFORT . ,effort) (CLOCKED . ,clocked) (SCHEDULED . ,scheduled) (DEADLINE . ,deadline) (DONE . ,done)))) (print task) (if (equal 'BAR type) (setq previous-end (decode-time deadline))) (insert (format-entry task)))) entries)) (insert (format "\\end{ganttchart}\n#+end_src"))))
The function works by mapping over the entries of the current tree and
processing each of them into either a \ganttgroup or \ganttbar. Each TODO
entry has a duration equal to the estimated effort of the task. If the effort
is not estimated, it is the same as 0, which pgfGantt shows as a day. While it
may be fine that a task defaults to a day, it does not properly show the likely
schedule given that all subsequent tasks start on the same day since there is
no duration to the tasks.
Usage
Add the following block to the subtree of entries.
#+begin: gantt :tstart "<today>" :tend "<2022-07-15>" :tunit "day" #+end:
Then executing org-update-dblock or pressing C-c C-c when the point is on the
block generates the necessary plotting commands, which when exported to PDF
generates the Gantt chart for the subtree. The parameters :tstart and :tend
can be any Org mode parseable date string, such as "<today>", "<yesterday>", or
some specific time stamp in ISO-8601 format. The unit parameter, :tunit can
either be "days" or "months".
Examples
To highlight how this function works, we consider the following example Org file. At the top level, there is an entry for all projects. Because this is an example, we have a single project named, "Major Project" which has 2 of its 5 major activities completed. We can see the generated Gantt chart for the major activity below. Within each of the activities, we can have arbitrarily deep subtasks. The following chart shows the breakdown of tasks for "Activity 3".
My apologies to dark mode readers, the image text elements do not invert in dark mode.
#+COLUMNS: %40ITEM(Task) %17Effort(Estimated Effort){:} %CLOCKSUM #+LATEX_HEADER: \usepackage{pgfgantt} #+LATEX_HEADER: \usepackage{fullpage} * Projects :PROPERTIES: :ID: 92c26e6f-327c-4517-ab73-000ab6b3794e :END: ** [2/5] Major Project DEADLINE: <2022-08-21 Sun> :PROPERTIES: :ID: bd0c7101-6df8-4fd3-838d-3b03a0d84677 :END: #+begin: gantt :tstart "<today>" :tend "<2022-08-21>" :tunit "month" #+end: *** DONE Activity 1 CLOSED: [2022-06-28 Tue 18:29] :PROPERTIES: :ID: 1ba87343-1926-49c3-8dc9-f01ffde95d05 :END: :LOGBOOK: - State "DONE" from "TODO" [2022-06-28 Tue 18:29] :END: **** DONE Task 1 CLOSED: [2022-06-28 Tue 18:32] :PROPERTIES: :ID: 6436a4d7-cc68-45fc-b46c-c10395b951a0 :END: :LOGBOOK: - State "DONE" from "TODO" [2022-06-28 Tue 18:32] :END: **** DONE Task 2 CLOSED: [2022-06-28 Tue 18:32] :PROPERTIES: :ID: cb294f7f-2e8d-49fc-9f53-b9fbdebd3f31 :END: :LOGBOOK: - State "DONE" from "TODO" [2022-06-28 Tue 18:32] :END: **** DONE Task 3 CLOSED: [2022-06-28 Tue 18:32] :PROPERTIES: :ID: 824d2572-830c-493c-bedc-b4cdcc8ce307 :END: :LOGBOOK: - State "DONE" from "TODO" [2022-06-28 Tue 18:32] :END: *** DONE Activity 2 CLOSED: [2022-06-28 Tue 18:29] :PROPERTIES: :ID: 81fb1423-447d-4fa3-92e4-cbba9d3aa7c8 :END: :LOGBOOK: - State "DONE" from "TODO" [2022-06-28 Tue 18:29] :END: *** TODO Activity 3 DEADLINE: <2022-07-28 Thu> :PROPERTIES: :ID: ce490c67-04f2-4060-b83d-f32675cc35e6 :EFFORT: 26d 0h 0min :END: #+begin: gantt :tstart "<today>" :tend "<2022-07-28>" :tunit "day" #+end: **** TODO Task 1 :PROPERTIES: :ID: f41e0083-7642-44a8-b653-f13124a1bcb9 :Effort: 5d :END: **** TODO Task 2 :PROPERTIES: :ID: e22aca56-18d9-4b43-b754-d11eadd73a97 :Effort: 7d :END: **** TODO Task 3 :PROPERTIES: :ID: c6bd2a10-dfcc-4e26-aa57-6629b6aa047c :Effort: 14d :END: **** TODO Task 4 :PROPERTIES: :ID: 78c88e86-da8a-41ba-991d-acfad713d4d1 :Effort: 4d :END: *** TODO Activity 4 :PROPERTIES: :ID: 6b92c6d0-ff88-4dbe-b9e9-da2b2e7b0fd9 :Effort: 14d :END: *** TODO Activity 5 :PROPERTIES: :ID: cabc7511-5430-42c6-86ea-f59899fc3875 :Effort: 14d :END:
Limitations
This process is currently pretty limited, see future work for improvements. One notable limitation, however, to make the charts fit on the page, the titles of the tasks are truncated to 15 characters. Furthermore, the code is fairly dense at the moment. It is certainly not the best code, but it serves the purpose. Perhaps with some more cycles, the quality and readability can be improved.
Future Work/Improvements
The tasks can be automatically linked via \ganttlinkedbar and this would be
natively supported by TODO dependencies. Alternatively, adding links could be
possible after parsing dependency information available via org-depend or
similar. Currently, neither of these are supported. The former would be
relatively straightforward to toggle by another parameter.
With respect to other visualizations, a burndown chart or even a process of generating evidence based schedules and velocity tables should be possible. Working on slow incremental changes to processes, going to start small.
Finally, there are always more ways to improve the usage of existing metadata
associated with the task. Currently, the SCHEDULED property is not used to
select a start date, but if it is set, it would act as a better start date than
whatever the previous task's end date is.