2008-09-24:
Perl Non-blocking Read On Pipes Or Files

Let's say you want to run a command in the "background" and check it occasionally (asynchronously) for full lines of output.

You can do a simple open with a pipe, but if you try to read the pipe it's blocking. And once you actually figure out how to do a non-blocking read, you get back chunks of data, not necessarily in happy complete lines of output, which is unfortunate if you want to, say, use regexps on the output.

This routine solves that problem by doing non-blocking reads on a filehandle and assembling the complete lines of output for you.

# An non-blocking filehandle read that returns an array of lines read
# Returns:  ($eof,@lines)
my %nonblockGetLines_last;
sub nonblockGetLines {
	my ($fh,$timeout) = @_;

	$timeout = 0 unless defined $timeout;
	my $rfd = '';
	$nonblockGetLines_last{$fh} = ''
		unless defined $nonblockGetLines_last{$fh};

	vec($rfd,fileno($fh),1) = 1;
	return unless select($rfd, undef, undef, $timeout)>=0;
	# I'm not sure the following is necessary?
	return unless vec($rfd,fileno($fh),1);
	my $buf = '';
	my $n = sysread($fh,$buf,1024*1024);
	# If we're done, make sure to send the last unfinished line
	return (1,$nonblockGetLines_last{$fh}) unless $n;
	# Prepend the last unfinished line
	$buf = $nonblockGetLines_last{$fh}.$buf;
	# And save any newly unfinished lines
	$nonblockGetLines_last{$fh} =
		(substr($buf,-1) !~ /[\r\n]/ && $buf =~ s/([^\r\n]*)$//) ? $1 : '';
	$buf ? (0,split(/\n/,$buf)) : (0);
}

I've tested it with a pipe I opened using an IO::File, YMMV. Example usage:

$fh = new IO::File; open($fh,"$cmd 2>&1 |");

do {
	($eof,@lines) = nonblockGetLines($fh);
	foreach my $line ( @lines ) {
		print "Pipe saw: [$line]\n";
	}
} until $eof;

close $fh;

I stole the select/vec code, and I haven't read up on vec() enough to know why it was called twice in the code I borrowed from, it may be an unnecessary call. If you have insight, let me know.

Enjoy!

Back to Solutions.