When developing applications, we often need to upload files to the cloud. Typically, Amazon S3 is the default choice for this task. However, S3 can be expensive and requires some configuration to set up properly.
With this in mind, I started looking for a more affordable alternative and discovered Backblaze B2. When comparing the pricing, Backblaze B2 is approximately one-fifth of the cost of AWS S3.
Backblaze B2 Pricing Comparison Although Backblaze B2 offers an S3-compatible API, this blog post will demonstrate how to upload files using their Native API .
Our Approach
In this post, we’ll create a flexible file storage system in Elixir that can work with both local storage and Backblaze B2. This approach is useful because we don’t always want to use cloud storage during development or testing. Here’s what we’ll cover:
- Setting up a storage behavior
- Implementing local storage
- Implementing Backblaze B2 storage
- How to use our new storage system
Defining the Storage Behavior
We’ll start by defining a behavior that our storage modules will implement. This approach allows us to switch between local and B2 storage without modifying our application code.
defmodule MyApp.Storage do
@callback write(path :: binary, value :: binary) :: :ok | {:error, term}
@callback read(path :: binary) :: {:ok, binary} | {:error, term}
end
Implementing Local Storage
Next, we’ll create a module for local storage, which is particularly useful for development and testing.
defmodule MyApp.Storage.Local do
@behaviour MyApp.Storage
require Logger
@impl true
def write(path, value) do
path = path(path)
File.mkdir_p!(Path.dirname(path))
File.write(path, value)
end
@impl true
def read(path) do
path = path(path)
case File.read(path) do
{:ok, contents} -> {:ok, contents}
{:error, :enoent} -> {:error, :not_found}
other -> other
end
end
defp path(path) do
Path.join([storage()[:path], path])
end
defp storage do
Application.get_env(:my_app, :storage)
end
end
Don’t forget to add the configuration to your application:
config :my_app,
storage: [
adapter: MyApp.Storage.Local,
path: "priv/my_storage"
]
Implementing Backblaze B2 Storage
Now, let’s implement the Backblaze B2 storage module. First, we’ll set up the configuration:
config :my_app,
storage: [
adapter: MyApp.Storage.Backblaze,
account_id: System.fetch_env!("BACKBLAZE_ACCOUNT_ID"),
application_key: System.fetch_env!("BACKBLAZE_APPLICATION_KEY"),
bucket_id: System.fetch_env!("BACKBLAZE_BUCKET_ID"),
bucket_name: System.fetch_env!("BACKBLAZE_BUCKET_NAME")
]
And here’s the Backblaze B2 module:
defmodule MyApp.Storage.Backblaze do
@behaviour MyApp.Storage
require Logger
@impl true
def write(path, value) do
with {:ok, auth_token, api_url} <- authorize(),
{:ok, upload_url, upload_auth_token} <- get_upload_url(auth_token, api_url),
:ok <- upload_file(upload_url, upload_auth_token, path, value) do
:ok
end
end
@impl true
def read(path) do
with {:ok, auth_token, api_url} <- authorize(),
{:ok, contents} <- download_file(auth_token, api_url, path) do
{:ok, contents}
else
{:error, :not_found} -> {:error, :not_found}
{:error, reason} -> {:error, reason}
end
end
# Helper functions
defp authorize do
url = "https://api.backblazeb2.com/b2api/v3/b2_authorize_account"
auth = Base.encode64("#{storage()[:account_id]}:#{storage()[:application_key]}")
case Req.get(url, headers: [{"Authorization", "Basic #{auth}"}]) do
{:ok, %{status: 200, body: %{"authorizationToken" => token, "apiInfo" => %{"storageApi" => %{"apiUrl" => url}}}}} ->
{:ok, token, url}
{:error, reason} ->
Logger.error("Failed to authorize with Backblaze B2: #{inspect(reason)}")
{:error, reason}
end
end
defp get_upload_url(auth_token, api_url) do
url = "#{api_url}/b2api/v3/b2_get_upload_url"
body = Jason.encode!(%{bucketId: storage()[:bucket_id]})
case Req.post(url, headers: [{"Authorization", auth_token}], body: body) do
{:ok, %{status: 200, body: %{"uploadUrl" => upload_url, "authorizationToken" => token}}} ->
{:ok, upload_url, token}
{:error, reason} ->
{:error, reason}
end
end
defp upload_file(upload_url, upload_auth_token, path, content) do
headers = [
{"Authorization", upload_auth_token},
{"X-Bz-File-Name", path},
{"Content-Type", "application/octet-stream"},
{"X-Bz-Content-Sha1", :crypto.hash(:sha, content) |> Base.encode16(case: :lower)}
]
case Req.post(upload_url, headers: headers, body: content) do
{:ok, %{status: 200}} ->
Logger.info("Successfully uploaded file to Backblaze B2: #{path}")
:ok
{:error, reason} ->
Logger.error("Failed to upload file to Backblaze B2: #{path}, reason: #{inspect(reason)}")
{:error, reason}
end
end
defp download_file(auth_token, api_url, path) do
download_url = "#{api_url}/file/#{storage()[:bucket_name]}/#{path}"
case Req.get(download_url, headers: [{"Authorization", auth_token}], decode_body: false) do
{:ok, %{status: 200, body: body}} -> {:ok, body}
{:ok, %{status: 404}} -> {:error, :not_found}
{:error, reason} -> {:error, reason}
end
end
defp storage do
Application.get_env(:my_app, :storage)
end
end
The Backblaze B2 upload process consists of three main steps:
- Authorize with the B2 API
- Get the upload URL
- Upload the file
The download process is similar, but it only requires getting the download URL from the B2 API.
Using the Storage System
With our storage modules in place, using them is straightforward:
@storage Application.get_env(:my_app, :storage)[:adapter]
@storage.write("path/to/my_file.txt", "Hello, world!")
@storage.read("path/to/my_file.txt")