Back
Featured image of post Upload a file with PowerShell and Invoke-RestMethod

Upload a file with PowerShell and Invoke-RestMethod

I came across the following challenge: Automate the upload of several build artefacts to the BitBucket cloud service downloads section. You might ask yourself “why is this a challenge”, and the answer was a little shock for me: “cause it is something that might get very complicated with Windows PowerShell 5”. I did some test on my Mac and I find it very easy with curl.

curl -X POST "https://MyUsername:MySectretPassword@api.bitbucket.org/2.0/repositories/dummyTeam/myproject/downloads" --form files=@"/home/dev/release\myproject-current.zip"

But curl is not available an my build server. So I tried to get it working with Invoke-RestMethod or Invoke-WebRequest. I ended up very frustrated! Searched around and found a lot of people that seem to have the same problem, but none of the answers seemed to work, at least not for me. Maybe an issue of the BitBucket cloud API, but I think it is more a general issue of Windows PowerShell.

So I started to do some tests and ended up with this function:

function Publish-BitbucketDownload
{
    <#
            .SYNOPSIS
          Upload given file to BitBucket cloud service downloads section.

            .DESCRIPTION
          Upload given file to BitBucket cloud service downloads section.
          I use this to upload build artifacts to the BitBucket Download section.

          The code might not be perfect, and we still use the AUTH Header instead of OAuth yet,
          but I needed a quick and dirty solution to get things going.

          I might change a few things soon, but for now; this function is doing what it should.

            .PARAMETER username
          BitBucket cloud username, as plain text

            .PARAMETER password
          BitBucket cloud password, as plain text

            .PARAMETER FilePath
          File to upload, full path needed

            .PARAMETER team
          BitBucket cloud team aka username (Might not be the login username!!!)

            .PARAMETER Project
          BitBucket cloud project name

            .EXAMPLE
          PS ~> Publish-BitbucketDownload -username 'MyUsername' -password 'MySectretPassword' -FilePath 'Y:\dev\release\myproject-current.zip' -team 'dummyTeam' -Project 'myproject'

          # Upload the artifact 'Y:\dev\release\myproject-current.zip' to the Download sections of the 'myproject' project of the 'dummyTeam', It uses User name and password (Both in plain ASC) to authenticate. However, both are converted to base64 to prevent any clear text header transfers.

            .EXAMPLE
          PS ~> Publish-BitbucketDownload -username 'MyUsername' -password 'MySectretPassword' -FilePath 'Y:\dev\release\myproject.nuget' -team 'dummyTeam' -Project 'myproject'

          # Upload the artifact 'Y:\dev\release\myproject.nuget' to the Download sections of the 'myproject' project of the 'dummyTeam', It uses Username and password (Both in plain ASC) to authenticate. However, both are converted to base64 to prevent any clear text header transfers.

            .NOTES
          I created this because I did not have CURL installed on my build system.

          With Curl this is an absolute no brainer:
          curl -X POST "https://MyUsername:MySectretPassword@api.bitbucket.org/2.0/repositories/dummyTeam/myproject/downloads" --form files=@"/home/dev/release\myproject-current.zip"

          INFO: Max. CPU: 16 %  Max. Memory: 28.48 MB

          TODO: Convert the request to use OAuth ASAP
  #>
	[CmdletBinding()]
	param
	(
		[Parameter(Mandatory = $true,
					  ValueFromPipeline = $true,
					  HelpMessage = 'BitBucket cloud username, as plain text')]
		[ValidateNotNullOrEmpty()]
		[Alias('user')]
		[string]
		$username,
		[Parameter(Mandatory = $true,
					  ValueFromPipeline = $true,
					  HelpMessage = 'BitBucket cloud password, as plain text')]
		[ValidateNotNullOrEmpty()]
		[Alias('pass')]
		[string]
		$password,
		[Parameter(Mandatory = $true,
					  ValueFromPipeline = $true,
					  HelpMessage = 'File to upload, full path needed')]
		[ValidateNotNullOrEmpty()]
		[string]
		$FilePath,
		[Parameter(Mandatory = $true,
					  ValueFromPipeline = $true,
					  HelpMessage = 'BitBucket cloud team name')]
		[ValidateNotNullOrEmpty()]
		[string]
		$team,
		[Parameter(Mandatory = $true,
					  ValueFromPipeline = $true,
					  HelpMessage = 'BitBucket cloud project name')]
		[ValidateNotNullOrEmpty()]
		[Alias('ProjectName')]
		[string]
		$Project
	)

	process
	{
		# Build the URI for our request
		$URI = 'https://api.bitbucket.org/2.0/repositories/' + $team + '/' + $Project + '/downloads'

		# Create our authentication header
		# TODO: Migrate to OAUTH
		$pair = ($username + ':' + $password)
		$encodedCreds = [Convert]::ToBase64String([Text.Encoding]::ASCII.GetBytes($pair))
		$basicAuthValue = ('Basic {0}' -f $encodedCreds)
		$Headers = @{
			Authorization = $basicAuthValue
		}

		# Cleanup the plain text stuff
		$pair = $null
		$encodedCreds = $null

		# The boundary is essential - Trust me, very essential
		$boundary = [Guid]::NewGuid().ToString()

        <#
              This is the crappy part: Build a body for a multipart request with PowerShell

              This is something that should be changed in PowerShell ASAP (I mean it is really crappy and really bad).

              It is an absolute no brainer with Curl.
      #>
		$bodyStart = @"
--$boundary
Content-Disposition: form-data; name="token"

--$boundary
Content-Disposition: form-data; name="files"; filename="$(Split-Path -Leaf -Path $FilePath)"
Content-Type: application/octet-stream


"@

		# Generate the end of the request body to finish it.
		$bodyEnd = @"

--$boundary--
"@

		# Now we create a temp file (Another crappy/bad thing)
		$requestInFile = (Join-Path -Path $env:TEMP -ChildPath ([IO.Path]::GetRandomFileName()))

		try
		{
			# Create a new object for the brand new temporary file
			$fileStream = (New-Object -TypeName 'System.IO.FileStream' -ArgumentList ($requestInFile, [IO.FileMode]'Create', [IO.FileAccess]'Write'))

			try
			{
				# The Body start
				$bytes = [Text.Encoding]::UTF8.GetBytes($bodyStart)
				$fileStream.Write($bytes, 0, $bytes.Length)

				# The original File
				$bytes = [IO.File]::ReadAllBytes($FilePath)
				$fileStream.Write($bytes, 0, $bytes.Length)

				# Append the end of the body part
				$bytes = [Text.Encoding]::UTF8.GetBytes($bodyEnd)
				$fileStream.Write($bytes, 0, $bytes.Length)
			}
			finally
			{
				# End the Stream to close the file
				$fileStream.Close()

				# Cleanup
				$fileStream = $null

				# PowerShell garbage collector
				[GC]::Collect()
			}

			# Make it multipart, this is the magic part...
			$contentType = 'multipart/form-data; boundary={0}' -f $boundary

            <#
                  The request itself is simple and easy, also works fine with Invoke-WebRequest instead of Invoke-RestMethod

                  I use Microsoft.PowerShell.Utility\Invoke-RestMethod to make sure the build in (Windows PowerShell native) function is used.
                  If PowerShell Core is installed or any Module provides a tweaked version... Just in case!
          #>
			try
			{
				$null = (Microsoft.PowerShell.Utility\Invoke-RestMethod -Uri $URI -Method Post -InFile $requestInFile -ContentType $contentType -Headers $Headers -ErrorAction Stop -WarningAction SilentlyContinue)
			}
			catch
			{
				# Remove the temp file
				$null = (Remove-Item -Path $requestInFile -Force -Confirm:$false)

				# Cleanup
				$contentType = $null

				# PowerShell garbage collector
				[GC]::Collect()

				# For the Build logs (will not break the build)
				Write-Warning -Message 'StatusCode:' $_.Exception.Response.StatusCode.value__
				Write-Warning -Message 'StatusDescription:' $_.Exception.Response.StatusDescription

				# Saved in the verbose logs for this build
				Write-Verbose -Message $_

				# Inform the build and terminate (Will break the build)
				Write-Error -Message 'We were unable to upload your file to the BitBucket downloads section, please check the build logs for further information.' -ErrorAction Stop
			}
		}
		finally
		{
			# Remove the temp file
			$null = (Remove-Item -Path $requestInFile -Force -Confirm:$false)

			# Cleanup
			$contentType = $null

			# PowerShell garbage collector
			[GC]::Collect()
		}
	}
}

There is also a Gist for this function.

Disclaimer:

  • This is not perfect! I know that, but I really needed something that could do the job for me, and this function works just fine!
  • I need to make the function a bit more robust, and I want to make use of OAuth instead of Username and Password. Deferred, time was an issue today.
  • I might publish a more general function in the future.