Hello World! Welcome back to the series where I show off the knowledge I am currently attaining about IT.

In the previous blog we learnt about how languages are encoded and best practices for a successful application, so now let's talk about digital images and sounds. After all, most people are either listening to music or watching videos on the internet.

Digital Audio

Sound comes from vibrations that cause sound waves. Computers convert sound waves into code by dividing the sound wave into samples. A sample is a single point on a sound wave. The computer converts the height of that sample to binary to recreate the wave.

Sound quality is determined by bit depth and sampling rate.

Bit Depth

Bit depth is how many bits are used to encode each sample.

Early computers used 8bit audio which meant there were only 256 possible values for each sample. 8-bit use one byte per sample while 16-bit use 2 bytes. Now we use 16-bit which make the range of sounds much higher (65,000 values) leading to more precise sounds.

Sampling Rate

Sampling rate is how many samples are taken per second. Think of a sample as a point on a graph. If you wanted to recreate a wave on a graph, the more points there are, (higher sample rate) the more precise you can graph the wave. After all, what the computer is doing is recreating what the sound wave looks like.

Digital Images

Displaying Images

Digital images are made up of tiny lights called an RGB Display.

RGB

RGB Displays are made up of many pixels. One pixel is a set of 3 lights; one red, one green, and one blue. These are primary colors and by changing how much of each color you use, you can create any color visible to humans.

##Storing Images There are two ways computers store Digital Images; Raster and Vector.

Raster Images

Raster, or bit map images are stored by the computer remembering the color of each pixel. The computer would store the RGB value and because they store each individual pixel's value they are very detailed. Your phone and digital camera creates Raster images. Most colors take 3 bytes 1 byte for each RGB value. This causes most Raster images to have huge file sizes. Another drawback of Raster images is that it does not scale well. The amount of pixels in that photo is not going to change so by zooming in you make the pixels larger and thus more blurry.

Vector Graphics

Instead of storing each individual pixel the computer stores points and curves. To make a square the computer would store the 4 edges and then connect them, and to add color you would simply need to tell the computer to fill in the shape. The greatest advantage to this is that it is scaleable so you will see it used for logos and such due to it being able to be used anywhere from billboards to business cards. One disadvantage to Vector files is that you need skill and certain tools to make them while Raster files you just need any camera. Another disadvantage is that you can not get it as detailed as Raster files.

Digital Videos

Every video is just a bunch of still images played in a rapid sequence. Video quality is determined by frame rate, resolution, and color depth.

Frame Rate

Frame rate is the measure of how many images play per second. The more images played per second the more smooth the video will look. Less will make it look choppy.

Resolution

Resolution is how many pixels are in a still image. The more pixels the higher the resolution and the sharper the image.

Color Depth

Color Depth is like Bit depth. Color Depth is the amount of memory used to display colors. 8-bit color use 1 byte of data per pixel meaning there are only 256 possible colors used. 24-bit uses 3 bytes per pixel this creates a much larger range of colors that can be used; Creating a more precise and sharper colored image.

Problem with High Quality

The main issue with having a higher bit depth and sample rate is that the file gets larger as you are using more binary. Until our compression gets better there really is no point in having higher quality sounds and videos as they will take forever to load.

File Compression

Compression is re-encoding a file in a way that uses fewer bits than the original.

Image Compression

There are two ways to compress files. Lossless Compression and Lossy Compression.

Lossless Compression

Lossless Compression is any form of compression that keeps the same information.

The file types that use Lossless Compression are GIF, PNG, PDF, and ZIP.

One way to compress images using lossless compression is Run-Length Encoding

Run-Length Encoding

This algorithm finds the colors that are the same and instead of using a bytes for each of them it just stores the colors RGB binary code and a value of how many times the color shows up.

Lossy Compression

Lossy Compression makes small changes to a file. One example of Lossy compression is changing similar rgb values to be the same value. This takes advantage of human sight not being extremely precise and you most likely will not be able to tell the difference.

The file types that use lossy compression are JPG, MP3, and MP4.

Video Compression

Because a video is just multiple images in a sequence you can use the same compression algorithms as for images.

Videos usually can compress further because frames often share the same background without changing. Compressing these pixels is called Temporal Redundancy.

Temporal Redundancy

Instead of retransmitting the reused pixels the algorithm just sends them to the next frame. This is the most common kind of compression and is the reason videos do not take days to upload.

Compressing Too Much

If a video is heavily compressed the computer may use temporal redundancy where it shouldn't have. Causing videos to appear laggy and pixelated. These over compressed areas are called Compression Artifacts.

Audio Compression

Audio compression usually uses lossy compression. Taking advantage of pitches that humans can not hear, it just simply discards the data that our ears would not process.

Phones compress the vocal ranges and non audible ranges which is why our voices sound more muffled on a phone call.

#Formatting Now that we have talked about compression, let's go over some file types that will best suit our needs.

Formatting your Audio

Before choosing which format is best for your audio file, you got to understand Bitrate. Bitrate is the amount of kilobits, per second, of audio data that is being processed. Lossy compression algorithms reduce bitrate Lower bitrate means reduced sound quality but smaller file sizes.

Here are some common audio file types and when you should use them.

WAV

If you are looking for quality audio I recommend WAV files as they are uncompressed. The tradeoff for better quality is that the file itself is much larger. Which like mentioned above makes sharing and streaming the audio take much longer.

MP3 and AAC

MP3 and AAC use lossy data compression. However, AAC has a more complex algorithm that allows it to have higher sound quality even though it is being compressed.

MIDI

For audio engineers, MIDI is the file type to use. Unlike the past file types, MIDI does not reproduce sound waves, instead MIDI translates instructions of which note is being played and for how long. MIDI is popular also because it can be easily modified by changing instructions for pitch and tempo.

##Formatting your Videos

Codecs

A codec is software that either decompresses or compresses videos. Codecs have two components, an encoder, and a decoder. The encoder compresses the video, when you save a video the codec using an encoder. The decoder, decompresses the video, when you press play on that saved file the computer uses a decoder.

Let's talk about some file types that are common codecs to format film.

MPEG-4

This uses an advanced compression algorithm that can compress a 2 hour film to just 300MB.

DivX

This is mainly used for DVDs, a little outdated I know. But it can compress and HD Movie onto a DVD.

Formatting your Video

While formatting your video you should take into account whether you value quality over file size.

MP4

One of the most common formats for videos is MP4 . Why you may ask? It has the ability to take up much less space than most file types. So based on that explanation it has to be a Compressed File. Not only does it work for videos but can also store audio, subtitles, and images included in the video. Almost all devices are configured to be able to play MP4

AVI

Unlike MP4, AVI only contains audio and video, not subtitles or images included in the video. This format is made up of Resource Interchange File Format files or RIFF for short. These RIFF files divide the AVI's data into smaller chunks. Each file contains two chunks. One chunk is some sort of MetaData about the file and the other chunk would be comprised of the actual visual and audio data. The Metadata is just instructions/information about the playback. For example one file would include frame rate and resolution as the first chunk and the other chunk would be the video itself.

AVI files are actually not compressed leading to very large files and can only run on Windows computers.

The final format I would like to mention and I am sure you have heard of it if you code on Mac, MOV files.

MOV

MOV was developed for Macs and is meant to function like MP4. MOV files contain a bunch of tracks that include video, text, audio, and time stamps. This file type, like MP4, is supported by QuickTime.

Differences between MOV and MP4

MP4 is a more universal format and has much more support than MOV. However, MOV has been know to be easier to edit.

Formatting Photos

Photos usually are what take up most of the space on your devices. Generally it is better to save them to the cloud online. If you do need to save them to your memory we are about to learn about some formats to best suit your needs.

JPG or JPEG

JPG is a compressed file type that reduces the file size by a ratio of 20:1 without sacrificing much image quality. And it supports 24-bit color. However it still does lose some image quality in the compression so it is considered a lossy compression.

PNG

PNG on the other hand is a raster file format that uses lossless compression. This was released to improve upon gif and jpg features. It also supports 24-bit color and was made with the purpose of allowing it to be shared online and used on most applications. You most likely have used this format when you needed a background removed. PNG has a feature called alpha channel that allows you to instruct different levels of transparency. These files cannot be animated unlike our next file type, GIF

GIF

GIFs store multiple images in a single file and display them one after another in an endless loop. The drawback to this is it has a limited color palette because it stores images using indexed colors. This makes images look grainy and unclear.

Now that we have learned about some 2D formatting let's go over some 3D media formatting.

3D Models

Remember how digital images can be stored as Vectors or Raster? 3D modeling graphics are all based on vectors. Meaning they scale well and are created using a multi-step process that involves wire framing, rendering, and tracing.

Wireframes

This is the first step in the modeling process. Think of a wire frame as the skeleton of the 3D model. A wire frame is the general shape of the object you want to create. Having a simple object is useful for rendering quickly; they are easy to manipulate and edit than complex models.

Rendering

Rendering is the process of converting 3D models into 2D images on a computer. The rendering process is often times when you add visual effects, set the scene, and add shading. The rendering process takes a lot of computing power so make sure you have your models exactly how you want it before rendering.

Ray Tracing

Ray tracing is an advanced rendering method that is used to create realistic lighting effects. This works by tracking every ray of light produced by a light source in the scene. The computer renders realistic light and shadow effects in real time. This requires powerful computers as the calculations are very complex.