Advanced Topics

Note: The materials of the this introduction are borrowed from ^[1]. Hence, it is highly recommended to read the Chapter 15 to Chapter 18 in ^[1].

Structs and objects

At this point you know how to use functions to organize code and built-in types to organize data. The next step is to learn how to build your own types to organize both code and data. This is a big topic; it will take a few chapters to get there.

Composite types

We have used many of Julia’s built-in types; now we are going to define a new type. As an example, we will create a type called Point that represents a point in two-dimensional space.

A programmer-defined composite type is also called a struct. For example, the struct definition for a point looks like this:

struct IPoint
    x
    y
end

A struct is like a factory for creating objects. To create a point, you call Point as if it were a function having as arguments the values of the fields. When Point is used as a function, it is called a constructor.

p = IPoint(3.0, 4.0)

IPoint(3.0, 4.0)

To access the attributes of p, we use the . operator:

p

IPoint(3.0, 4.0)

Structs are immutable

As we can see from below, a Point is an immutable type

ismutable(p)

false

To define a mutable struct, we use the keyword mutable struct in place of struct:

mutable struct MPoint
    x
    y
end

mp = MPoint(1, 2)

MPoint(1, 2)

mp.x = 3

3

struct Rectangle
    width::Real
    height::Real
    corner
end

Structs and functions

Pure functions and modifiers

A pure function is a function without side effects (e.g. modifying the arguments) after execution and the returned value only on the input passed to the function.

Here is an example of a pure function:

A modifier function, on the other hand, modifies the the its input and returns nothing. The name of a modifier function, in Julia, is conventially ends with a exclamation mark !. Learn more about this from the documentation.

Here is an example of a modifier:

function reset_point(p::MPoint)
    (p.x, p.y)=(0, 0)
    return nothing
end

reset_point (generic function with 1 method)

let
    p=MPoint(1, 2)
    reset_point(p)
    println((p.x, p.y))
end

(0, 0)

Function overloading

The idea of function overloading is that you can define two functions taking different arguments with the same name. For example, we can difine a distance function calculating two different points:

Constructors

A constructor is a special function that is called to create an object. The default constructor methods of Rectangle have the following signatures:

Rectange(width::Real, height::Real, corner)

They simply initialize the attribute of the object. We can also define our own inner constructor, by using new in a these constructors to generate a new object. An example is given as follows:

mutable struct Square
    side
    corner
    #' customized constructor (override the default constructor)
    function Square(side, corner = IPoint(0, 0))
        @assert(side >= 0, "Side length is negative!")
        return new(side, corner)
    end
    #' copy constructor
    function Square(square::Square)
        new(square.side, square.corner)
    end
end

We can compare the following results:

let
    square1 = Square(3, IPoint(0, 0))
    square2 = Square(square1)
    (square1 == square2), (square1 === square2)
end

(false, false)

let
    square1 = Square(3, IPoint(0, 0))
    square2 = square1
    (square1 == square2), (square1 === square2)
end

(true, true)

In the code above, we have a constructor taking an argument of type Square. It is useful when we decide to make a generate a new copy of the object later on.

show

We now override the show function to generate an string expression for type Square

function Base.show(io::IO, p::MPoint)
    print(io, "($(p.x), $(p.y))")
end

function Base.show(io::IO, p::IPoint)
    print(io, "($(p.x), $(p.y))")
end

function Base.show(io::IO, square::Square)
    print(io, "Square anchored at $(square.corner) with side length $(square.side)")
end

Square(2, IPoint(0, 0))

Square anchored at (0, 0) with side length 2

Square(2, MPoint(0, 0))

Square anchored at (0, 0) with side length 2

Operator overloading and multiple dispatch

Some operators in Julia, like other functions, can be overloaded. These operators are in Base module, and operator op in the module should referred to as Base:op or even Base:(op) (e.g. mandatory for ==)

function Base.:+(p::MPoint, s::Square)
    t = Square(s)
    s.corner.x += p.x
    s.corner.y += p.y
    return t
end

distance(p) = sqrt(p.x^2 + p.y^2)

distance (generic function with 1 method)

distance(p, q) = sqrt((p.x - q.x)^2 + (p.y - q.y)^2)

distance (generic function with 2 methods)

let
    p=MPoint(1, 2)
    println(distance(p))
    println((p.x, p.y))
end

2.23606797749979
(1, 2)

let
    p=MPoint(1, 2)
    println(distance(p), distance(p, p))
end

2.236067977499790.0

MPoint(3, 4) + Square(3, MPoint(1, 2))

Square anchored at (4, 6) with side length 3

The choice of which method to execute when a function is applied is called dispatch.

Generic programming

Many of the functions we wrote for strings also work for other sequence types. For example, in Dictionary as a Collection of Counters we used histogram to count the number of times each letter appears in a word.

function histogram(s)
    d = Dict()
    for c in s
        if c ∉ keys(d)
            d[c] = 1
        else
            d[c] += 1
        end
    end
    d
end

histogram (generic function with 1 method)

histogram(("spam", "egg", "spam", "spam", "bacon", "spam"))

Dict{Any, Any} with 3 entries:
  "bacon" => 1
  "spam" => 4
  "egg" => 1

histogram(["spam", "egg", "spam", "spam", "bacon", "spam"])

Dict{Any, Any} with 3 entries:
  "bacon" => 1
  "spam" => 4
  "egg" => 1

Functions that work with several types are called polymorphic. Polymorphism can facilitate code reuse.

Abstract types and subtyping

In the following, we use the same example as ^[1] to demonstrate abstract type and subtyping.

Here is our setup

const suit_names = ["♣", "♦", "♥", "♠"]

4-element Vector{String}:
 "♣"
 "♦"
 "♥"
 "♠"

const rank_names = ["A", "2", "3", "4", "5", "6", "7", "8", "9", "10", "J", "Q", "K"]

13-element Vector{String}:
 "A"
 "2"
 "3"
 "4"
 "5"
 "6"
 "7"
 "8"
 "9"
 "10"
 "J"
 "Q"
 "K"

struct Card
    suit :: Int64
    rank :: Int64
    #' constructor
    function Card(suit::Int64, rank::Int64)
        @assert(1 ≤ suit ≤ 4, "suit is not between 1 and 4")
        @assert(1 ≤ rank ≤ 13, "rank is not between 1 and 13")
        new(suit, rank)
    end
    #' methods overloading
    function Base.show(io::IO, card::Card)
        print(io, rank_names[card.rank], suit_names[card.suit])
    end
    #' operator overloading
    function Base.:<(c1::Card, c2::Card)
        (c1.suit, c1.rank) < (c2.suit, c2.rank)
    end
end

We now define a virtual type CardSet, to which both a deck and a hand belong to:

abstract type CardSet end

struct Hand <: CardSet
    cards :: Array{Card, 1}
    label :: String
    function Hand(label::String="")
        Hand(Card[], label)
    end
end

We can defining generic functions on CardSet, which apply to both decks and hands:

begin
    function Base.show(io::IO, cs::CardSet)
        for card in cs.cards
            print(io, card, " ")
        end
    end

    function Base.pop!(cs::CardSet)
        pop!(cs.cards)
    end

    function Base.push!(cs::CardSet, card::Card)
        push!(cs.cards, card)
        nothing
    end
end

struct Deck <: CardSet
    cards :: Array{Card, 1}
    #' constructor
    function Deck()
        deck = new(Card[])
        for suit in 1:4
            for rank in 1:13
                push!(deck.cards, Card(suit, rank))
            end
        end
        deck
    end
end

Deck()

A♣ 2♣ 3♣ 4♣ 5♣ 6♣ 7♣ 8♣ 9♣ 10♣ J♣ Q♣ K♣ A♦ 2♦ 3♦ 4♦ 5♦ 6♦ 7♦ 8♦ 9♦ 10♦ J♦ Q♦ K♦ A♥ 2♥ 3♥ 4♥ 5♥ 6♥ 7♥ 8♥ 9♥ 10♥ J♥ Q♥ K♥ A♠ 2♠ 3♠ 4♠ 5♠ 6♠ 7♠ 8♠ 9♠ 10♠ J♠ Q♠ K♠ 

deck = Deck()

A♣ 2♣ 3♣ 4♣ 5♣ 6♣ 7♣ 8♣ 9♣ 10♣ J♣ Q♣ K♣ A♦ 2♦ 3♦ 4♦ 5♦ 6♦ 7♦ 8♦ 9♦ 10♦ J♦ Q♦ K♦ A♥ 2♥ 3♥ 4♥ 5♥ 6♥ 7♥ 8♥ 9♥ 10♥ J♥ Q♥ K♥ A♠ 2♠ 3♠ 4♠ 5♠ 6♠ 7♠ 8♠ 9♠ 10♠ J♠ Q♠ K♠ 

Others

repr

Create a string from any value using the show function. You should not add methods to repr; define a show method instead.

repr(zip("abc", [1 2 3]))

"zip(\"abc\", [1 2 3])"

dump

Show every part of the representation of a value.

dump(zip("abc", [1 2 3]))

Base.Iterators.Zip{Tuple{String, Matrix{Int64}}}
  is: Tuple{String, Matrix{Int64}}
    1: String "abc"
    2: Array{Int64}((1, 3)) [1 2 3]

:: operator

We may use a :: T to assert if a variable a is of type T:

let
    f(x::Int,y::Int) = x+y
    try
        println(f(1.0, 2.0))
    catch e
        println("Error: $e")
    end
    try
        println(f(1, 2))
    catch e
        println("Error: $e")
    end
end

Error: MethodError(Main.FD_SANDBOX_5514522325621768181.var"#f#1"(), (1.0, 2.0), 0x0000000000007b61)
3

It may also help us to perform assignment:

let
    x::Int8 = round(Int, 2.0)
    typeof(x)
end

Int8

Macro call '@'

A macro maps a tuple of arguments, expressed as space-separated expressions or a function-call-like argument list, to a returned expression. A macro named macro_name can be called by @macro_name.

:: operator

:expr quote an expression expr, returning the abstract syntax tree (AST) of expr. The AST may be of type Expr, Symbol, or a literal value. The syntax :identifier evaluates to a Symbol.

Named tuple

You can name the components of a tuple, creating a named tuple:

x = (a=1, b=1+1)

(a = 1, b = 2)

@isdefined x

true

x.a

1

do blocks

In Reading and Writing we had to close the file after when where done writing. This can be done automatically using a do block:

let
    data = "This here's the wattle,\nthe emblem of our land.\n"
    open("output.txt", "w") do fout
        write(fout, data)
    end
end

48

In this example fout is the file stream used for output. This is functionally equivalent to

let
    data = "This here's the wattle,\nthe emblem of our land.\n"
    f = fout -> begin
        write(fout, data)
    end
    open(f, "output.txt", "w")
end

48

"The way it works is the following:"

"The way it works is the following:"

function open(f::Function, args...)
    io = open(args...)
    try
        f(io)
    finally
        close(io)
    end
end

Check out the documentation.

Ternary operator condition ? true : false

The is an alternative to an if-elseif statement:

1 > 0 ? "It is true!" : "It is false!"

"It is true!"

Short-Circuit Evaluation

The operators && and || do a short-circuit evaluation: a next argument is only evaluated when it is needed to determine the final value.

function fact(n::Integer)
    n >= 0 || error("n must be non-negative")
    n == 0 && return 1
    n * fact(n-1)
end

fact (generic function with 1 method)

fact(4)

24

Primitive, composite, abstract and parametric types

Primitive types are a concrete types whose data consists of plain old bits, and their identity depend only on bits.

Composite type are called records, structs, or objects in various languages. A composite type is a collection of named fields, an instance of which can be treated as a single value. In many languages, composite types are the only kind of user-definable type, and they are by far the most commonly used user-defined type in Julia as well.

Primitive/composite/abstract types can have parameters. See the following example:

struct Point{T<:Real}
    x::T
    y::T
end

Type unions

isa(8, Union{Real, String})

true

isa("s", Union{Int64, String})

true

Parametric Methods

Method definitions can also have type parameters qualifying their signature:

isintpoint(p::Point{T}) where {T} = (T === Int64)

isintpoint (generic function with 1 method)

isintpoint(Point(1, 2))

true

Function-like Objects

Any arbitrary Julia object can be made “callable”. Such “callable” objects are sometimes called functors.

let
    struct Polynomial{R}
        coeff::Vector{R}
    end
    function (p::Polynomial)(x)
        val = p.coeff[end]
        for coeff in p.coeff[end-1:-1:1]
            val = val * x + coeff
        end
        val
    end
    p = Polynomial([1,10,100])
    p(3)
end

931

Constructors

Parametric types can be explicitly or implicitly constructed:

Point(1,2)         # implicit T

Point{Int64}(1, 2)

Point{Int64}(1, 2) # explicit T

Point{Int64}(1, 2)

Point(1,2.5)       # implicit T

MethodError: no method matching Main.FD_SANDBOX_5514522325621768181.Point(::Int64, ::Float64)

Closest candidates are:
  Main.FD_SANDBOX_5514522325621768181.Point(::T, !Matched::T) where T<:Real
   @ Main.FD_SANDBOX_5514522325621768181 none:2

To address the above ambiguous issue, we can use promote function:

begin
    struct PointMultitype{T<:Real}
        x::T
        y::T
        PointMultitype{T}(x::T,y::T) where {T<:Real} = new{T}(x,y)
        PointMultitype(x::Real, y::Real) = Point(promote(x,y)...);
    end
    PointMultitype(1,2.5)
end

Point{Float64}(1.0, 2.5)

Conversion and Promotion

Julia has a system for promoting arguments to a common type. This is not done automatically but can be easily extended.

We can add our own convert methods:

let
    Base.convert(::Type{Point{T}}, x::Array{T, 1}) where {T<:Real} = Point(x...)
    convert(Point{Int64},[1,2])
end

Point{Int64}(1, 2)

On the other hand, promotion is the conversion of values of mixed types to a single common type:

promote(1, 2.5, 3)

(1.0, 2.5, 3.0)

Methods for the promote function are normally not directly defined, but the auxiliary function promote_rule is used to specify the rules for promotion:

promote_rule(::Type{Float64}, ::Type{Int32}) = Float64

promote_rule (generic function with 1 method)

Metaprogramming

Julia code can be represented as a data structure of the language itself. This allows a program to transform and generate its own code.

Every Julia program starts as a string, which we can parse into an object called an expression:

begin
    ex = Meta.parse("1 + 2") # expression 1
    ex = quote # expression 2
        1 + 2
    end
end

quote
    #= none:4 =#
    1 + 2
end

typeof(ex)

Expr

dump(ex)

Expr
  head: Symbol block
  args: Array{Any}((2,))
    1: LineNumberNode
      line: Int64 4
      file: Symbol none
    2: Expr
      head: Symbol call
      args: Array{Any}((3,))
        1: Symbol +
        2: Int64 1
        3: Int64 2

We can use eval to evaluate expressions:

eval(ex)

3

Macros

Macros can include generated code in a program. A macro maps a tuple of Expr objects directly to a compiled expression:

macro containervariable(container, element)
    return esc(:($(Symbol(container,element)) = $container[$element]))
end

@containervariable (macro with 1 method)

The macro call @containervariable letters 1 is replaced by :(letters1 = letters[1]).

Measuring Performance

@time sum([1 2 3 4])

  0.000002 seconds (1 allocation: 96 bytes)
10

Interactive Utilities

@code_lowered sum([1 2 3 4])

CodeInfo(
1 ─      nothing
│   %2 = Base.:(:)
│   %3 = Core.NamedTuple()
│   %4 = Base.pairs(%3)
│   %5 = Base.:(var"#sum#807")(%2, %4, #self#, a)
└──      return %5
)

@code_typed sum([1 2 3 4])

CodeInfo(
1 ─ %1 = Base.identity::typeof(identity)
│   %2 = Base.add_sum::typeof(Base.add_sum)
│   %3 = invoke Base._mapreduce(%1::typeof(identity), %2::typeof(Base.add_sum), $(QuoteNode(IndexLinear()))::IndexLinear, a::Matrix{Int64})::Int64
└──      return %3
) => Int64

@code_llvm sum([1 2 3 4])

;  @ reducedim.jl:994 within `sum`
; Function Attrs: uwtable
define i64 @julia_sum_338({}* noundef nonnull align 16 dereferenceable(40) %0) #0 {
top:
; ┌ @ reducedim.jl:994 within `#sum#807`
; │┌ @ reducedim.jl:998 within `_sum`
; ││┌ @ reducedim.jl:998 within `#_sum#809`
; │││┌ @ reducedim.jl:999 within `_sum`
; ││││┌ @ reducedim.jl:999 within `#_sum#810`
; │││││┌ @ reducedim.jl:357 within `mapreduce`
; ││││││┌ @ reducedim.jl:357 within `#mapreduce#800`
; │││││││┌ @ reducedim.jl:365 within `_mapreduce_dim`
          %1 = call i64 @j__mapreduce_340({}* nonnull %0) #0
; └└└└└└└└
  ret i64 %1
}

@code_native sum([1 2 3 4])

.text
        .file   "sum"
        .globl  julia_sum_368                   # -- Begin function julia_sum_368
        .p2align        4, 0x90
        .type   julia_sum_368,@function
julia_sum_368:                          # @julia_sum_368
; ┌ @ reducedim.jl:994 within `sum`
        .cfi_startproc
# %bb.0:                                # %top
        pushq   %rbp
        .cfi_def_cfa_offset 16
        .cfi_offset %rbp, -16
        movq    %rsp, %rbp
        .cfi_def_cfa_register %rbp
        subq    $32, %rsp
; │┌ @ reducedim.jl:994 within `#sum#807`
; ││┌ @ reducedim.jl:998 within `_sum`
; │││┌ @ reducedim.jl:998 within `#_sum#809`
; ││││┌ @ reducedim.jl:999 within `_sum`
; │││││┌ @ reducedim.jl:999 within `#_sum#810`
; ││││││┌ @ reducedim.jl:357 within `mapreduce`
; │││││││┌ @ reducedim.jl:357 within `#mapreduce#800`
; ││││││││┌ @ reducedim.jl:365 within `_mapreduce_dim`
        movabsq $j__mapreduce_370, %rax
        callq   *%rax
; │└└└└└└└└
        addq    $32, %rsp
        popq    %rbp
        retq
.Lfunc_end0:
        .size   julia_sum_368, .Lfunc_end0-julia_sum_368
        .cfi_endproc
; └
                                        # -- End function
        .section        ".note.GNU-stack","",@progbits

Compare these with @which

@which sum([1 2 3 4])

sum(a::AbstractArray; dims, kw...)
     @ Base reducedim.jl:994

@debug macro

let
    x=5
    y=-5
    while x > 0 && y < 0
        x -= 1
        y += 1
        @debug "variables" x y
        @debug "condition" x > 0 && y < 0
    end
end

┌ Debug: variables
│   x = 4
│   y = -4
└ @ Main REPL[10]:7
┌ Debug: condition
│   x > 0 && y < 0 = true
└ @ Main REPL[10]:8
┌ Debug: variables
│   x = 3
│   y = -3
└ @ Main REPL[10]:7
┌ Debug: condition
│   x > 0 && y < 0 = true
└ @ Main REPL[10]:8
┌ Debug: variables
│   x = 2
│   y = -2
└ @ Main REPL[10]:7
┌ Debug: condition
│   x > 0 && y < 0 = true
└ @ Main REPL[10]:8
┌ Debug: variables
│   x = 1
│   y = -1
└ @ Main REPL[10]:7
┌ Debug: condition
│   x > 0 && y < 0 = true
└ @ Main REPL[10]:8
┌ Debug: variables
│   x = 0
│   y = 0
└ @ Main REPL[10]:7
┌ Debug: condition
│   x > 0 && y < 0 = false
└ @ Main REPL[10]:8

References

• [1]	Think Julia: How to Think Like a Computer Scientist, https://benlauwens.github.io/ThinkJulia.jl/latest/book.html.