# Neural network tutorial

It's possible to define a neural network as a surrogate, using Flux. This is useful because we can call optimization methods on it.

First of all we will define the Schaffer function we are going to build surrogate for.

using Plots
using Surrogates
using Flux

function schaffer(x)
x1=x[1]
x2=x[2]
fact1 = x1 ^2;
fact2 = x2 ^2;
y = fact1 + fact2;
end
schaffer (generic function with 1 method)

## Sampling

Let's define our bounds, this time we are working in two dimensions. In particular we want our first dimension x to have bounds 0, 8, and 0, 8 for the second dimension. We are taking 60 samples of the space using Sobol Sequences. We then evaluate our function on all of the sampling points.

n_samples = 60
lower_bound = [0.0, 0.0]
upper_bound = [8.0, 8.0]

xys = sample(n_samples, lower_bound, upper_bound, SobolSample())
zs = schaffer.(xys);
60-element Array{Float64,1}:
56.65625
40.65625
22.65625
12.65625
80.65625
38.65625
52.65625
3.90625
55.90625
68.90625
⋮
31.0078125
126.0078125
39.0078125
39.0078125
8.0078125
71.0078125
55.5078125
31.5078125
8.0078125

## Building a surrogate

You can specify your own model, optimization function, loss functions and epochs. As always, getting the model right is hardest thing.

model1 = Chain(
Dense(1, 5, σ),
Dense(5,2,σ),
Dense(2, 1)
)
neural = NeuralSurrogate(x, y, lb, ub, model = model1, n_echos = 10)

## Optimization

We can now call an optimization function on the neural network:

surrogate_optimize(schaffer, SRBF(), lower_bound, upper_bound, neura, SobolSample(), maxiters=20, num_new_samples=10)