MatrixInd_1
Download script
This script functions as a data processing tool designed to organize and standardize sets of numerical values (attributes) into a format suitable for machine learning-style analysis. Instead of calculating traditional price indicators, it focuses on the mathematical "normalization" of datasets.
Core Logic Overview
The script operates by managing two distinct groups of data: a Sample Set (often used as a reference or training set) and a Test Set (the data being actively processed). It treats these datasets as matrices—grids of numbers where each column represents a different characteristic or attribute.
Key Functional Steps
1. Data Generation and Initialization When the script runs, it generates randomized numerical values for both the Sample and Test sets. These represent raw data points that need to be "leveled" so they can be compared accurately against one another.
2. The Normalization Engine The heart of the script is its ability to normalize data. In statistics, normalization ensures that different variables are on a comparable scale (for example, ensuring that a variable ranging from 1–10 and another from 1,000–10,000 can be compared equally). The script offers four distinct ways to perform this:
- Standard Deviation Method: It calculates the "average" and the "spread" (standard deviation) of the data. It then adjusts the numbers so they center around zero with a consistent spread.
- Range and Midrange Method: It finds the highest and lowest values in a set to determine the total "width" of the data. It then shifts the numbers so that the middle point sits at zero.
3. Comparative Processing Modes The script is highly flexible in how it applies these rules, offering different modes for how the Test Set should be treated relative to the Sample Set:
- Independent Scaling: The Test Set is normalized using only its own internal math.
- Dependent Scaling: The Test Set is "forced" to fit the scale of the Sample Set (using the Sample Set's average, spread, or range). This is a common technique in data science to ensure that new data conforms to the patterns established by historical data.
Performance Behavior
The script is designed to be efficient by only performing these complex calculations when a new price bar appears on the chart (specifically on the 1-minute timeframe). Once the math is completed, it outputs three distinct visual lines (colored Yellow, Red, and LightSeaGreen) in a separate window, representing the processed results of the three different attributes.
2
Views
0
Downloads
0
Favorites
MatrixInd_1
//+------------------------------------------------------------------+
//| MatrixInd_1.mq4 |
//| Copyright © 2008, MetaQuotes Software Corp. |
//| http://www.metaquotes.net |
//+------------------------------------------------------------------+
#property copyright "Copyright © 2008, MetaQuotes Software Corp."
#property link "http://www.metaquotes.net"
#property indicator_separate_window
//#property indicator_chart_window
#property indicator_buffers 3
#property indicator_color1 Yellow
#property indicator_color2 Red
#property indicator_color3 LightSeaGreen
#property indicator_level1 0.0
#define LIN1 100 //sample lines
#define LIN2 100 //test lines
#define COL 3 //attributes
extern bool normalize_bufers = true;
extern int normalize_type=1;
double M_test_attr[LIN2][COL];
double M_sample_attr[LIN1][COL];
double Buffer1[];
double Buffer2[];
double Buffer3[];
//+------------------------------------------------------------------+
//| Custom indicator initialization function |
//+------------------------------------------------------------------+
int init()
{
//---- indicators
SetIndexStyle(0,DRAW_LINE);
SetIndexBuffer(0,Buffer1);
SetIndexStyle(1,DRAW_LINE);
SetIndexBuffer(1,Buffer2);
SetIndexStyle(2,DRAW_LINE);
SetIndexBuffer(2,Buffer3);
//----
return(0);
}
//+------------------------------------------------------------------+
//| Custom indicator deinitialization function |
//+------------------------------------------------------------------+
int deinit()
{
//----
//----
return(0);
}
//+------------------------------------------------------------------+
//| Custom indicator iteration function |
//+------------------------------------------------------------------+
int start()
{
int counted_bars=IndicatorCounted();
//----
if(isNewBar1(PERIOD_M1))
{
main();
}
//----
return(0);
}
//+------------------------------------------------------------------+
//+------------------------------------------------------------------+
//| Custom indicator iteration function |
//+------------------------------------------------------------------+
int myBars1;
bool isNewBar1(int tf1)
{
//----
bool res=false;
if (myBars1!=iBars(Symbol(),tf1))
{
res=true;
myBars1=iBars(Symbol(),tf1);
}
//----
return(res);
}
void main()
{
int i,j;
for(i=0;i<LIN1;i++)
for(j=0;j<COL;j++)
M_sample_attr[i][j]=MathRand();
for(i=0;i<LIN2;i++)
for(j=0;j<COL;j++)
M_test_attr[i][j]=MathRand();
if(normalize_bufers)
svm_normalize_test(M_sample_attr,M_test_attr,normalize_type);
for(i=0; i < LIN2;i++)
{
Buffer1[i]=M_test_attr[i][0];
Buffer2[i]=M_test_attr[i][1];
Buffer3[i]=M_test_attr[i][2];
}
}
/////////////////////////////////////////////////////////////////
#define DBL_MAX 99999999999999999999
//+------------------------------------------------------------------+
//|
//+------------------------------------------------------------------+
double MAX(double x, double y)
{
if(x>y)
return(x);
else
return(y);
}
//+------------------------------------------------------------------+
//|
//+------------------------------------------------------------------+
double MIN(double x, double y)
{
if(x<y)
return(x);
else
return(y);
}
//+------------------------------------------------------------------+
//| Normalize matrix 'fmx' using precomputed 'xmean', 'std'. The
//| normalization is: for each column, subtract 'xmean' and, if the
//| column 'std' is non-zero, divide by 'std'.
//| d=nr_of_attrib
//| nv=nr_of_sample_lines
//| double **x --> double **svm_samples_attrib_scaled_dyn_
//| double *xmean = fmx_mean(x2, nv, d);
//| double *std = fmx_std(x2, nv, d);
//| fmx_prenorm(x, nv, d, xmean, std);
//| X - mean
//| i
//| S = ---------
//| i std
//+------------------------------------------------------------------+
void fmx_prenorm(
double &fmx[][],
double xmean[], // double *xmean = fmx_mean(x2, nv, d);
double std[] // double *std = fmx_std(x2, nv, d);
)
{
int i;
int j;
double f;
int rows = ArrayRange(fmx,0);
int columns = ArrayRange(fmx,1);
for (i = 0; i < columns; i++)
for (j = 0; j < rows; j++)
{
f = fmx[j][i];
if (std[i] != 0)
fmx[j][i] = (f-xmean[i])/std[i];
else
fmx[j][i] = f-xmean[i];
}
}
//+------------------------------------------------------------------+
//| max X + min X
//| i i i i
//| midrange = ----------------
//| 2
//| range = max X - min X
//| i i i i
//|
//| X - midrange
//| i
//| S = -------------
//| i range / 2
//+------------------------------------------------------------------+
void fmx_prenorm_range(
double &fmx[][],
double xrange[],
double xmidrange[]
)
{
int i;
int j;
double f;
int rows = ArrayRange(fmx,0);
int columns = ArrayRange(fmx,1);
for (i = 0; i < columns; i++)
for (j = 0; j < rows; j++)
{
f = fmx[j][i];
if (xrange[i] != 0)
fmx[j][i] = (f-xmidrange[i])/(xrange[i]/2);
else
fmx[j][i] = f-xmidrange[i];
}
}
//+------------------------------------------------------------------+
//|
//+------------------------------------------------------------------+
void fmx_midrange(double &xmidrange[],double fmx[][])
{
int i;
int j;
double smax = -DBL_MAX;
double smin = DBL_MAX;
int rows = ArrayRange(fmx,0);
int columns = ArrayRange(fmx,1);
ArrayResize(xmidrange,columns);
for (i = 0; i < columns; i++)
{
smax = -DBL_MAX;
smin = DBL_MAX;
for (j = 0; j < rows; j++)
{
smax=MAX(fmx[j][i],smax);
smin=MIN(fmx[j][i],smin);
}
xmidrange[i] = (smax+smin)/2;
}
}
//+------------------------------------------------------------------+
//|
//+------------------------------------------------------------------+
void fmx_range(double &xrange[], double fmx[][])
{
int i;
int j;
double smax = -DBL_MAX;
double smin = DBL_MAX;
int rows = ArrayRange(fmx,0);
int columns = ArrayRange(fmx,1);
ArrayResize(xrange,columns);
for (i = 0; i < columns; i++)
{
smax = -DBL_MAX;
smin = DBL_MAX;
for (j = 0; j < rows; j++)
{
smax=MAX(fmx[j][i],smax);
smin=MIN(fmx[j][i],smin);
}
xrange[i] = smax-smin;
}
}
//+------------------------------------------------------------------+
//|
//+------------------------------------------------------------------+
void fmx_max(double &xrange[],double fmx[][])
{
int i;
int j;
double smax = -DBL_MAX;
int rows = ArrayRange(fmx,0);
int columns = ArrayRange(fmx,1);
ArrayResize(xrange,columns);
for (i = 0; i < columns; i++)
{
smax = -DBL_MAX;
for (j = 0; j < rows; j++)
{
smax=MAX(fmx[j][i],smax);
}
xrange[i] = smax;
}
}
//+------------------------------------------------------------------+
//|
//+------------------------------------------------------------------+
void fmx_min(double &xrange[],double fmx[][])
{
int i;
int j;
double smin = DBL_MAX;
int rows = ArrayRange(fmx,0);
int columns = ArrayRange(fmx,1);
ArrayResize(xrange,columns);
for (i = 0; i < columns; i++)
{
smin = DBL_MAX;
for (j = 0; j < rows; j++)
{
smin=MIN(fmx[j][i],smin);
}
xrange[i] = smin;
}
}
//+------------------------------------------------------------------+
//| Return mean values of columns in 'fmx'.
//+------------------------------------------------------------------+
void fmx_mean(double &xmean[], double fmx[][])
{
int i;
int j;
double s;
int rows = ArrayRange(fmx,0);
int columns = ArrayRange(fmx,1);
ArrayResize(xmean,columns);
if (rows > 1)
{
for (i = 0; i < columns; i++)
{
s = 0.0;
for (j = 0; j < rows; j++)
s += fmx[j][i];
xmean[i] = s/rows;
}
}
else
{
for (i = 0; i < columns; i++)
xmean[i] = fmx[0][i];
}
}
//+------------------------------------------------------------------+
//| Set elements of 'vec' to 'value'.
//+------------------------------------------------------------------+
void fvec_set(double &vec[], int len, double value)
{
int i;
ArrayResize(vec,len);
for (i = 0; i < len; i++)
vec[i] = value;
}
//+------------------------------------------------------------------+
//| Return standard deviations of columns in 'fmx'.
//+------------------------------------------------------------------+
void fmx_std(double &std[], double fmx[][])
{
int i;
int j;
double xmean;
double s;
int rows = ArrayRange(fmx,0);
int columns = ArrayRange(fmx,1);
ArrayResize(std,columns);
if (rows > 1)
{
for (i = 0; i < columns; i++)
{
s = 0.0;
for (j = 0; j < rows; j++)
s += fmx[j][i];
xmean = s/rows;
s = 0.0;
for (j = 0; j < rows; j++)
s += (fmx[j][i]-xmean)*(fmx[j][i]-xmean);
std[i] = MathSqrt(s/(rows-1));
}
}
else
fvec_set(std, columns, 0.0);
}
//+------------------------------------------------------------------+
//| 1-normalize TRAIN lines from std and mean calculated from TRAIN lines
//| 1-normalize TEST lines from std and mean calculated from TEST lines
//| ..................................................................
//| 2-normalize TRAIN lines from std and mean calculated from TRAIN lines
//| 2-normalize TEST lines from std and mean calculated from TRAIN lines
//| ..................................................................
//| 3-normalize TRAIN lines from range and midrange calculated from TRAIN lines
//| 3-normalize TEST lines from range and midrange calculated from TEST lines
//| ..................................................................
//| 4-normalize TEST lines from range and midrange calculated from TRAIN lines
//+------------------------------------------------------------------+
void svm_normalize_sample(
double &svm_samples_attrib[][],
int normalize
)
{
double xmean[];
double std[];
double xrange[];
double xmidrange[];
if(normalize==1 || normalize==2)
{ //normalize TRAIN lines from std and mean calculated from TRAIN lines
fmx_mean(xmean,svm_samples_attrib);
fmx_std(std,svm_samples_attrib);
fmx_prenorm(svm_samples_attrib,xmean, std);
}
else if(normalize==3)
{ //normalize TRAIN lines from range and midrange calculated from TRAIN lines
fmx_range(xrange,svm_samples_attrib);
fmx_midrange(xmidrange,svm_samples_attrib);
fmx_prenorm_range(svm_samples_attrib,xrange, xmidrange);
}
}
//+------------------------------------------------------------------+
//| 1-normalize TEST lines from std and mean calculated from TEST lines
//| 2-normalize TEST lines from std and mean calculated from TRAIN lines
//| 3-normalize TEST lines from range and midrange calculated from TEST lines
//| 4-normalize TEST lines from range and midrange calculated from TRAIN lines
//+------------------------------------------------------------------+
void svm_normalize_test(
double svm_samples_attrib[][],
double &svm_test_attrib[][],
int normalize
)
{
if(normalize==1)
{ //normalize TEST lines from std and mean calculated from TEST lines
double xmean2[];
fmx_mean(xmean2,svm_test_attrib);
double std2[];
fmx_std(std2,svm_test_attrib);
fmx_prenorm(svm_test_attrib,xmean2, std2);
}
else if(normalize==2)
{ //normalize TEST lines from std and mean calculated from TRAIN lines
double xmean1[];
fmx_mean(xmean1,svm_samples_attrib);
double std1[];
fmx_std(std1,svm_samples_attrib);
fmx_prenorm(svm_test_attrib, xmean1, std1);
}
else if(normalize==3)
{ //normalize TEST lines from range and midrange calculated from TEST lines
double xrange2[];
fmx_range(xrange2,svm_test_attrib);
double xmidrange2[];
fmx_midrange(xmidrange2,svm_test_attrib);
fmx_prenorm_range(svm_test_attrib, xrange2, xmidrange2);
}
else if(normalize==4)
{ //normalize TEST lines from range and midrange calculated from TRAIN lines
double xrange1[];
fmx_range(xrange1,svm_samples_attrib);
double xmidrange1[];
fmx_midrange(xmidrange1,svm_samples_attrib);
fmx_prenorm_range(svm_test_attrib, xrange1, xmidrange1);
}
}
Comments