š¦ My First Contribution to the Official Rust Repo š¦
January 11, 2026 marks the day that the first PR I made to the Rust repository officially got merged into the main branch. :)
Context
The PR I made revolves around std::fs::create_dir_all() function. To quote what the function does:
Recursively create a directory and all of its parent components if they are missing.
This function is not atomic. If it returns an error, any parent components it was able to create will remain.
If the empty path is passed to this function, it always succeeds without creating any directories.
Itās essentially similar to what the mkdir -p <path> command does in the terminal.
Funnily enough, the reason why I made this PR was because of a PR I oversaw in uutilsā coreutils repository on fixing a stack overflow issue with mkdir -p on deeply nested directories. Prior to this change for the mkdir command, uutilsā coreutils manually recursed up to the first non-existing path component and used std::fs::create_dir to create each missing directory:
// Return true if the directory at `path` has been created by this call.
// `is_parent` argument is not used on windows
#[allow(unused_variables)]
fn create_dir(path: &Path, is_parent: bool, config: &Config) -> UResult<()> {
let path_exists = path.exists();
...
if path == Path::new("") {
return Ok(());
}
if config.recursive {
match path.parent() {
Some(p) => create_dir(p, true, config)?,
None => {
USimpleError::new(1, translate!("mkdir-error-failed-to-create-tree"));
}
}
}
match std::fs::create_dir(path) {
Ok(()) => {
...
Ok(())
}
Err(_) if path.is_dir() => Ok(()),
Err(e) => Err(e.into()),
}
}
As naoNao89 pointed out in the PR, the implementation of uutilsā mkdir command had room for a stack overflow issue with nested directories (in his testing, the overflow occurred at 200+ nested directories of ā/aā repeated). The solution to this problem is to change the implementation of mkdir -p from creating directories and missing components from recursive to iterative; from allocating the path references in the stack to the heap.
Still, when I first took a look at how uutilsā initially implemented the mkdir command with -p flag set, I thought āwhy didnāt they just use std::fs::create_dir_all() instead of std::fs::create_dir()?ā to create all the directory components with the recursive flag set. There would be no need to re-invent the wheel on creating the given directory and its missing parent components. On second glance, however, it clicked for me. For mkdir, it has other config options like permission bits to set on the directory once it is created. If std::fs::create_dir_all() were to report an error but successfully created some directories, then it could leave the directories in an incorrect state if config options like permission bits were set.
But surely, ignoring the config options for a moment, std::fs::create_dir_all() would have avoided this stack overflow issue, right?
Rustās std::fs::create_dir_all() Implementation
That would be an incorrect assumption to make! The following is the implementation of Rustās create_dir_all():
path#[stable(feature = "rust1", since = "1.0.0")]
pub fn create_dir_all<P: AsRef<Path>>(path: P) -> io::Result<()> {
DirBuilder::new().recursive(true).create(path.as_ref())
}
impl DirBuilder {
...
#[stable(feature = "dir_builder", since = "1.6.0")]
pub fn recursive(&mut self, recursive: bool) -> &mut Self {
self.recursive = recursive;
self
}
#[stable(feature = "dir_builder", since = "1.6.0")]
pub fn create<P: AsRef<Path>>(&self, path: P) -> io::Result<()> {
self._create(path.as_ref())
}
fn _create(&self, path: &Path) -> io::Result<()> {
if self.recursive { self.create_dir_all(path) } else { self.inner.mkdir(path) }
}
fn create_dir_all(&self, path: &Path) -> io::Result<()> {
if path == Path::new("") {
return Ok(());
}
match self.inner.mkdir(path) {
Ok(()) => return Ok(()),
Err(ref e) if e.kind() == io::ErrorKind::NotFound => {}
Err(_) if path.is_dir() => return Ok(()),
Err(e) => return Err(e),
}
match path.parent() {
Some(p) => self.create_dir_all(p)?,
None => {
return Err(io::const_error!(
io::ErrorKind::Uncategorized,
"failed to create whole tree",
));
}
}
match self.inner.mkdir(path) {
Ok(()) => Ok(()),
Err(_) if path.is_dir() => Ok(()),
Err(e) => Err(e),
}
}
...
}
Funny side note: the None block for match path.parent() is kind of useless because the .parent() only returns None on root paths ("/") or prefix, which this should already return Ok(()) or an error by the self.inner.mkdir() earlier
The standard library implementation of Rustās create_dir_all(), as promised by the doc comments, uses recursion!
Whatās Wrong with Recursion Here?
For all this talk about recursion causing issues, I guess I should go a bit deeper on why recursion may not be so pleasant for this function.
Because the nature of DirBuilderās create_dir_all() function is recursive, &self and path arguments are stored onto the stack frame repeatedly since itās possible that the parent directories of the given path needs to be created before the current path can be created; itās necessary to remember the paths you need to create, hence the allocation is required. This also means Tail Call Optimization (TCO) is not possible here, which means that the recursive block of code cannot be transformed into an iterative version to prevent allocating a new stack frame.
If we were to follow conventional path size limits (filesystems inherently do not have limits on how deeply nested a path is), then recursively creating directories would not pose an issue on:
Linux-based OSs, which often has a default stack size of 8 MiB and a
PATH_MAXof 4096 bytes;MacOS, which has a default stack size of 8 MiB and maximum path of 1024 bytes;
Windows OS, which has a default stack size of 1 MB and a maximum path of 260 bytes
However, if we consider enabling long paths on Windows OS and multithreading across all OSs, then we would need to be aware of the following:
Linux-based OSs tends to to default to 8 MB per thread;
MacOS defaults to 512 KB stacks per thread (unrelating to main thread);
And Windows OS allows paths up to 32767 bytes with long path enabled and defaults to 1 MB stacks per thread
On MacOS and Windows OS, itās definitely possible for a potential stack overflow to occur through multithreaded code or with long paths on Windows. In fact, if we take this code
use std::fs;
fn main() {
let mut dir_path = "".to_string();
for letter in 0..16000 {
dir_path = format!("{}{}", dir_path, "/a");
}
fs::create_dir_all(&dir_path);
}
and output the debug build assembly in Rustās playground for std::fs::create_dir_all()
std::fs::DirBuilder::create:
subq $72, %rsp
movq %rdi, 8(%rsp)
movq %rsi, 32(%rsp)
movq %rdx, 40(%rsp)
movq %rdi, 48(%rsp)
leaq 32(%rsp), %rdi
callq <&T as core::convert::AsRef<U>>::as_ref
movq %rdx, 16(%rsp)
movq %rax, 24(%rsp)
jmp .LBB1_3
.LBB1_1:
movq 56(%rsp), %rdi
callq _Unwind_Resume@PLT
movq %rax, %rcx
movl %edx, %eax
movq %rcx, 56(%rsp)
movl %eax, 64(%rsp)
jmp .LBB1_1
.LBB1_3:
movq 16(%rsp), %rdx
movq 24(%rsp), %rsi
movq 8(%rsp), %rdi
movq std::fs::DirBuilder::_create@GOTPCREL(%rip), %rax
callq *%rax
movq %rax, (%rsp)
jmp .LBB1_4
.LBB1_4:
movq (%rsp), %rax
addq $72, %rsp
retq
std::fs::create_dir_all:
subq $72, %rsp
movq %rdi, 32(%rsp)
movb $1, 55(%rsp)
movl $511, 44(%rsp)
movb $0, 48(%rsp)
movb $1, 48(%rsp)
leaq 32(%rsp), %rdi
callq <&T as core::convert::AsRef<U>>::as_ref
movq %rdx, 16(%rsp)
movq %rax, 24(%rsp)
jmp .LBB2_3
.LBB2_1:
movq 56(%rsp), %rdi
callq _Unwind_Resume@PLT
movq %rax, %rcx
movl %edx, %eax
movq %rcx, 56(%rsp)
movl %eax, 64(%rsp)
jmp .LBB2_1
.LBB2_3:
movq 16(%rsp), %rdx
movq 24(%rsp), %rsi
leaq 44(%rsp), %rdi
callq std::fs::DirBuilder::create
movq %rax, 8(%rsp)
jmp .LBB2_4
.LBB2_4:
movq 8(%rsp), %rax
addq $72, %rsp
retq
From the instruction subq $72, %rsp, we can observe the following that on Linux systems, each recursive call to std::fs::create_dir_all() takes up 72 bytes of the stack frame. Without inspecting deeply into assembly output, we can infer that this stack allocation is reserved for things like a reference to a path to mkdir on, some permission bits on creation of a directory (as 511 in decimal is 777 in octal, which is used to give rwx permissions across owner, group, and others), and return addresses. Considering shadow space padding on Windows, each recursive call to create_dir_all() could easily take up in the ballpark of ~72-112 bytes. That means on Windows, if we were to try and create a path of ā/aā repeated, the stack would overflow around ~8929-13889 function calls deep. In practical cases, itās very rare to reach this deep in this function, but it exists in the realm of possibility!
The Fix to std::fs::create_dir_all()
The key to solving this issue is just like what was proposed in naoNao89ās PR: just make the function create directories iteratively. To be precise, we should turn DirBuilderās create_dir_all() into an iterative function. The following is the implementation change to DirBuilderās create_dir_all():
fn create_dir_all(&self, path: &Path) -> io::Result<()> {
// if path's parent is None, it is "/" path, which should
// return Ok immediately
if path == Path::new("") || path.parent() == None {
return Ok(());
}
let ancestors = path.ancestors();
let mut uncreated_dirs = 0;
for ancestor in ancestors {
// for relative paths like "foo/bar", the parent of
// "foo" will be "" which there's no need to invoke
// a mkdir syscall on
if ancestor == Path::new("") || ancestor.parent() == None {
break;
}
match self.inner.mkdir(ancestor) {
Ok(()) => break,
Err(e) if e.kind() == io::ErrorKind::NotFound => uncreated_dirs += 1,
// we check if the err is AlreadyExists for two reasons
// - in case the path exists as a *file*
// - and to avoid calls to .is_dir() in case of other errs
// (i.e. PermissionDenied)
Err(e) if e.kind() == io::ErrorKind::AlreadyExists && ancestor.is_dir() => break,
Err(e) => return Err(e),
}
}
// collect only the uncreated directories w/o letting the vec resize
let mut uncreated_dirs_vec = Vec::with_capacity(uncreated_dirs);
uncreated_dirs_vec.extend(ancestors.take(uncreated_dirs));
for uncreated_dir in uncreated_dirs_vec.iter().rev() {
if let Err(e) = self.inner.mkdir(uncreated_dir) {
if e.kind() != io::ErrorKind::AlreadyExists || !uncreated_dir.is_dir() {
return Err(e);
}
}
}
Ok(())
}
In order to preserve the behavior and time complexity of recursive version of create_dir_all() to the iterative version, I had to keep in mind that:
Root path (
ā/ā) and empty string (āā) must just return with anOk(())We must iterate upward to the first non-existing path to create and then we can iterate downward to create the missing path components
The errors returned by
self.inner.mkdir()produced have different effects.io::ErrorKind::NotFoundtells us we need to create a missing parent component firstio::ErrorKind::AlreadyExists && ancestor.is_dir()tells us that we have reached the first ancestor directory that exists and we can start building the missing components thereEvery other error is a valid error we should return impromptu
We should only allocate onto the heap the required space to contain the remaining paths that we need to create
The last point is pretty subtle on why itās necessary (or more so efficient than necessary). In the recursive implementation of create_dir_all(), you can see that no extra allocation or re-copying is done for the path and parent components. If we used an empty Vec and kept pushing the paths that we need to mkdir on repeatedly, we would need to resize the Vec repeatedly to allow for more paths to be pushed onto the Vec; there would be unnecessary re-allocations of heap memory occurring here. Hence, a counter variable is used to track how many directories we have left to create, and we can then take from the ancestor iterator the exact number of directories and place that onto our Vec.
In an ideal world, we can make this function better and not need to use heap allocation for the remaining directories we need to create. We could just use the ancestor iterator directly and with the counter variable we can just create the given path and missing parent directories in a forward manner. But unfortunately, the Ancestor<ā_> iterator returned by .ancestors() is not a reversible iterator; repeatedly going backward from the iterator to create the missing directories would result in a O(NĀ²) time complexity instead of O(N). That said, this implementation matches the time and space complexity of the recursive version of create_dir_all().
Afterwords
To be honest, itās a bit surreal that a PR that I made got merged into the Official Rust Repo. Iāve only been programming in Rust as my primary language since summer 2024 (prior to that I mostly worked with Python and C since 2021), so thereās a lot of things that I still need to learn and get comfortable with, like lifetimes and proc macro. On top of that, I started contributing to open source software/projects during last October (uutilsā coreutils being my first!), so a lot of conventions of following contributing guidelines and communicating with people through GitHub were really new to me. So when I made this PR back in October, I thought a change like this might not be looked at or be merged in.
But seeing that this PR got merged in, it made me happy. Iām happy that the code I write can have impact on peopleā that I could even cause that sort of impact. Iām really glad to be a part of the open source community, and especially be a part of Rust community who have all been super cool to interact with and inspiring to see what projects theyāve worked on.
I hope to be able to contribute more to the open source community and write more blog posts!