Home QuickTipps $Input : Reading Pipeline Input in one Chunk
 

$Input : Reading Pipeline Input in one Chunk

Sometimes, it becomes necessary for a function to first collect all pipeline data before processing it. Sorting would be such a task. You may have heard about $Input, but there are some not-so-documented secrets about it that were introduced in PowerShell 3.0. Let’s take a look at how the $Input automatic variable really works and how it can help you write better code. Let’s also look at how you can make the new features available in PowerShell 2.0 as well, producing compatible code.

Collecting Pipeline Data

Sometimes, people add a lot of code just to collect and store pipeline input. Scripts may look like this:

  

function Test-Function
{
  param
  (
    [Parameter(ValueFromPipeline=$true)]
    $InputObject
  )
 
  begin
  {
    $container = @()
  }
 
  process
  {
    $container += $InputObject
  }
 
  end
  {
    $count = $container.Count
    "Collected $count elements."
  }
}

  A lot of code to receive pipeline data

When someone pipes data into the function, it does collect all items and can then process them in one chunk:

PS> 1..100 | Test-Function
Collected 100 elements.

This would count the number of received objects (of course there are better ways for counting, this is just to illustrate how you collect data).

This is a lot of code for a simple task, and it is inefficient code, too. It uses an array to collect data which needs to get expanded each time a new element is added.

Automatic Data Collector

You could enhance this by, for example, using an ArrayList object as collector, but beginning withPowerShell 3.0, things become a lot easier. The automatic variable $Input can now turn into an automatic data collector and reduces the code to this:

  

function Test-Function
{
  param
  (
    [Parameter(ValueFromPipeline=$true)]
    $InputObject
  )
 
  $count = $Input.Count
  "Collected $count elements."
}

  A lot more efficient, but requires PSv3 or better

This function produces the exact same result as the one before, with a lot less code, and it is a lot faster, too. Unfortunately, it only works well in PowerShell 3.0. In PowerShell 2.0, $Input is not an array but just a so-called enumerator. Enumerators have no property Count, and you can read its contents only once.

A function for all PowerShell versions

Fortunately, it is very easy to make this function compatible to older versions of PowerShell, because all you need to do is turn $Input into an array. This is basically what PowerShell 3.0 does by default. Here is a function that works equally well in all versions of PowerShell:

  

function Test-Function
{
  param
  (
    [Parameter(ValueFromPipeline=$true)]
    $InputObject
  )
 
  $alldata = @($Input)
  $count = $alldata.Count
  "Collected $count elements."
}

  This function works in all PowerShell versions

Note how $Input is enclosed in @() which turns it into an array if it wasn’t an array before. Now you can read its Count property and read its contents as often as you wish.

Always Remember: Use $Input plus a Pipeline Parameter!

You may have seen scripts that use $Input without declaring a pipeline-aware parameter. That’s not a good thing to do. One reason is that without a pipeline-aware parameter, $Input will always be an enumerator only, not an array (even in PowerShell 3.0). That’s not too much of a problem anymore, though, because you have seen that enclosing $Input in @() fixes the problem and produces a real array.

What’s worse is usability. A function that only uses $Input can only receive data over the pipeline. A user would not be able to submit the data without a pipeline because there simply is no parameter.

The Price You Pay For Collecting Things

Aside from all of this, always remember that $Input is collecting pipeline data. So you get the data only after the pipeline completed. That may take some time, and it costs memory. Use it only if you must have simultaneous access to all pipeline results. In all other cases, make your function process pipeline data in realtime, using a process block.

In fact, PowerShell will collect pipeline data in $Input only as long as it assumes that you do not want to process the pipeline data in real time. So the moment you introduce a process block to your function code, $Input changes nature again. Now, $Input represents the currently incoming data inside your process block. In your end block, $Input is now empty.

Things To Keep in Mind

Use “blocking” functions (functions that collect pipeline data) with care and only where appropriate. In most scnearios, it is not necessary to collect all incoming pipeline data before processing it. In these scenarios, process incoming pipeline data individually inside a process block. This saves memory and provides better user experience.

When you use $Input to collect pipeline data, do not use a process block. Do add a parameter to your function that accepts pipeline input.

Enclose $Input in @() to turn it into a real array. Else, it may just be a simple enumerator which can only be read once (unless you call its method Reset()).