Each solution vector in the algorithm is a particle moving due to the combined effect of the force of inertia, memory and acceleration reported by the best particle. The coordinates of the ith particle at the time of iteration t are determined by the vector x i ( t ) = ( x i 1 ( t ) , x i 2 ( t ) , … , x i D ( t ) ), where i = 1 , 2 , … , N, N is a particle count in a swarm. All particles store their best position p i ( t ) = ( p i 1 ( t ) , p i 2 ( t ) , … , p i D ( t ) ) for the iterations they have passed. The velocity of a particle is set by a vector v i ( t ) = ( v i 1 ( t ) , v i 2 ( t ) , … , v i D ( t ) ). The speed and position of the ith particle at t + 1 iteration are determined by the following expressions: (1) x i ( t + 1 ) = x i ( t ) + v i ( t + 1 ) , (2) v i ( t + 1 ) = w × v i ( t ) + c 1 × rand 1 × ( p i ( t ) − x i ( t ) ) + c 2 × rand 2 × ( p g ( t ) − x i ( t ) ) , where c 1 , c 2 are positive acceleration coefficients; p i ( t ) is a vector of previous coordinates of the particle x i ( t ) that had the best value of the fitness function, p g ( t ) is a vector of coordinates of the particle that reached the best value of the fitness function in the entire swarm for t iterations; w is the empirical coefficient of inertia; rand 1 and rand 2 are random numbers from the interval [ 0 , 1 ]. The swarm stops moving when at least one of the following conditions is met: the swarm has reached a state of equilibrium, an optimal solution has been found, or a specified number of iterations has been performed.

Feature selection can be represented as an NP-hard binary optimization problem (Kohavi and John, 1997). The most optimal solution of this problem can be guaranteed to be found only by a brute-force search. However, brute-force approach is ineffective for solving this problem, since the size of the feature space in modern datasets can amount to several tens or even hundreds. Going through all possible combinations of such number of elements is a very time- and resource-consuming task. The decision to this problem can be the use of metaheuristic algorithms that allow finding a suboptimal solution in an acceptable time.

Most metaheuristic algorithms are designed to operate in a continuous search space. However, binary search space may be considered as a more suitable way to deal with feature selection task due to its structure. Accordingly, in order to use metaheuristics for addressing the feature selection task in binary search space, their modification is required. The modification should allow the algorithm to function in a binary space, while maximally preserving the original idea of metaheuristics. There are many ways to binarize metaheuristic algorithms. The main ones will be considered below: the method of modified algebraic operations, the method of transformation functions, and the quantum method.

1) Modified algebraic operations: The idea of the method is that to convert the algorithm to the binary version, it is enough to put their logical analogues in accordance with the original mathematical operations of the formulas used. For example, to use disjunction (OR) instead of addition, conjunction (AND) instead of multiplication, exclusive or (XOR) instead of subtraction, and logical negation (NOT). Thus, the search space of the algorithm changes from continuous to binary, but the original idea of changing the position of particles is formally preserved.

However, there are two disadvantages that make it difficult to apply this method to some algorithms. The first is that the method is practically inapplicable if the algorithm uses any operations other than addition, subtraction, multiplication, and division. The second disadvantage is typical for all methods that require working with binary vectors. Many metaheuristic algorithms, in addition to operations directly on vectors, also use a variety of coefficients represented by real numbers. When switching to binary vectors, these coefficients must also be changed or abandoned, which leads to a change in the original idea of the algorithm and, accordingly, may affect the quality of its work. So, when modifying the PSO algorithm, the authors of Yuan et al. (2009) refused to use the inertia coefficient, and used randomly generated binary vectors as speed coefficients.

Despite the external similarity of the algorithm binarized by the method of modified algebraic operations with the original continuous algorithm, its results are not always satisfactory. In such cases, the authors of some works use only a part of logical operations or use their own modified versions instead of some “classic” logical operations. In Kıran and Gündüz (2012) when binarizing the artificial bee colony algorithm, the authors analysed the use of logical operations and came to the conclusion that it is necessary to use only two of them: XOR and NOT. In this case, the NOT operation was modified as follows: a random real number from 0 to 1 was generated, and if it was greater than 0.5, a logical negation was performed, otherwise the vector remained unchanged.

2) Transfer functions: Transformation functions are often used to switch from a continuous space to a binary one. They convert a real number into the range between zero and one. If the calculated function value is greater than a random number, then the corresponding element of the vector either takes the value of one, or is replaced with the opposite (Mirjalili and Lewis, 2013).

This method avoids the limitations of the previous method, since it does not require any changes to the original algorithm. However, this method has another complexity, which consists in the problem of choosing a specific function as a transformation one. The research (Mirjalili and Lewis, 2013), which examined six transformation functions from two families (S-shaped and V-shaped), showed that the use of V-shaped functions significantly improves the performance of the PSO algorithm. In Souza et al. (2020), the PSO algorithm, binarized with a V-shaped transformation function, was used to select features when solving the problem of static verification of a handwritten signature. S-shaped and V-shaped transform functions have also been successfully applied to binarize gray wolf algorithm (Chantar et al., 2020), bat swarm (Qasim and Algamal, 2020), dragonfly algorithm (Mafarja et al., 2017), and butterfly swarm (Arora and Anand, 2019). In Ghosh et al. (2020), a new X-shaped transformation function, consisting of two S-shaped ones, was proposed and applied to binarize the Social Mimic algorithm. With this approach, one vector of real numbers is converted into two binary vectors, from which the best is selected.

3) Quantum Method: The idea of this method is to use vectors represented by Q-bits instead of binary vectors, and principles that apply to quantum mechanics, such as quantum superposition. In Hamed et al. (2009), it was shown that using the quantum method for binarization of the PSO algorithm allows increasing the speed of searching for the optimal set of features. In Barani et al. (2017), the authors also explored a quantum approach for binarization of the gravitational search algorithm when solving the feature selection task using the kNN classifier. The principle of superposition was used in the work, and a quantum rotation gate was applied to update the solutions. The researchers concluded that the proposed algorithm can significantly reduce the total number of features, while maintaining a comparable classification accuracy relative to other algorithms. The authors (Ashry et al., 2020) compared binary gray wolf algorithms using the quantum method (BQI-GWO) and the S-shaped transformation function (BGWO). Comparison of the algorithms showed that the BQI-GWO makes it possible to exclude a larger number of features, while increasing the classification accuracy. Researchers in Zouache and Ben Abdelaziz (2018) proposed a hybrid swarm intelligence algorithm based on quantum computations and a combination of the firefly algorithm and PSO for feature selection. Quantum computations provided a good tradeoff between the intensification and diversification of search, while the combination of the firefly algorithm and PSO made it possible to efficiently investigate the generated subsets of features. To evaluate the relevance of these subsets, rough set theory was employed.

The set of instances is a list of objects of the type x i = { x i 1 , x i 2 , … , x i D }, where x i j is a value of the jth feature of the ith object, j ∈ { 1 , 3 , … , D }, D is a number of features, i ∈ { 1 , 2 , … , N }, N is a number of objects. The task of classification is to match each object with the most suitable class from the set of all classes c = { c 1 , c 2 , … , c K }, K is a number of classes. Decision algorithms restore the dependence between input attributes and classes.

In this paper, k-nearest neighbour (KNN) is used as a decision algorithm, as one of the simplest and most frequently used classifiers. The algorithm has only one parameter k that indicates the number of neighbours used to define a class label for a new data instance. As a result of the absence of other parameters, the algorithm does not need a learning stage or setting of these parameters, which is expressed in its high speed.

The algorithm operation process can be described as follows: 1.

To determine the value k.

The feature selection problem is formulated as follows: on the given set of features, find a feature subset that does not cause a significant decrease in classification accuracy, or even increases it, when the number of features is decreased. The solution is represented as a vector s = { s 1 , s 2 , … , s D }, where s i = 0 means that the ith feature is excluded from classification and s i = 1 means that the ith feature is used by the classifier.

Since an algorithm that works as a wrapper has been chosen to select features, a fitness function that evaluates the quality of a subset of selected variables must be used. We have applied the following function: (3) Fit = α × Error + ( 1 − α ) × F / D , where Error is a classification error, F is a number of features in the evaluated subset, α is the priority coefficient between an error and the ratio of selected features, α ∈ [ 0 , 1 ]. The classification error is the number of incorrectly classified objects in relation to the total number of instances.

The task of the binary optimization algorithm is to search for the minimum of the given function.

Since first founded by Shannon (1948) to address communication, information theory has now been implemented to a variety of fields, from clustering, fuzzy logic to decision making, just to name a few. Specifically, information theory has been implemented to quantify information. The information quantity is described as the amount of information observed in an event (called uncertainty) computed using probabilistic measures. In addition to the probabilistic measures, the fuzziness measures are also very common as an uncertainty measure among researchers. In particular, the fuzziness measures differ from the probabilistic measures by introducing vagueness uncertainties rather than randomness uncertainties.

The first fuzziness extension of the Shannon’s entropy was first considered by De Luca and Termini (1972). Specifically, the fuzzy entropy was described on the basis of a membership function. Based on the axioms of De Luca and Termini, Al-Ani and Khushaba (2012) defined the membership of kth feature belonging to ith class as follows: (4) μ i k = ( ‖ x i ˆ − x k ‖ σ r ± ϵ ) − 2 m − 1 , where m is the fuzziness, σ is the standard deviation, r is the radius of the data such that r = max ‖ x i ˆ − x k ‖ σ , x i ˆ is the mean of data samples belonging to ith class.

Based on the membership function defined by Eq. (4), the memberships of all data samples (with size NP) for each of fuzzy sets (with size C) along kth and lth features are respectively denoted as A and B. In other words, A and B are C × NP membership matrixes for kth and lth features, respectively. Based on the membership matrix, the fuzzy marginal uncertainty measure along each kth feature is defined as follows: (5) P ( k ) = A A ′ N P . The fuzzy marginal entropy along each kth feature is then calculated as follows: (6) H ( k ) = − ∑ ∑ P ( k ) log P ( k ) . The fuzzy joint entropy, called H ( k , l ), between kth and lth features is calculated as follows: (7) H ( k , l ) = − ∑ ∑ A B ′ N P log A B ′ N P . Using the joint entropy, the fuzzy mutual information between kth and lth features can be calculated as follows: (8) I ( k , l ) = H ( k ) + H ( l ) − H ( k , l ) .

Assuming S is a selected feature subset for a dataset. The intra-class feature dependencies (denoted as relevancy (Rel)) and the inter-feature dependencies (denoted as redundancy (Red)) are respectively defined as follows: (9) R e l ( S ) = ∑ k ∈ S I ( k , y ) , (10) R e d ( S ) = ∑ k ∈ S ∑ l ∈ S I ( k , l ) , where y denotes the output variable. Then, in terms of the intra-class and the inter-feature dependencies, the fuzzy version of MIFS (Battiti, 1994) along S can be calculated as follows: (11) FMIFS = γ × R e l + ( 1 − γ ) × Red , where γ is a priority coefficient between Rel and Red.

In case of fuzzy entropy measures, the fuzzy version of symmetry uncertainty (SU) (Witten and Frank, 2002) between the kth feature and the output variable can be calculated as follows: (12) FSU ( k , y ) = H ( k ) + H ( k | y ) H ( k ) + H ( y ) , where H ( k | y ) represents the conditional entropy of the output variable given kth feature.

This subsection describes the investigated modifications of the PSO algorithm for finding suboptimal subsets of features. A population of randomly generated binary vectors s is given to the algorithm input. After passing a specified number of iterations, the PSO outputs a solution with the best value of the fitness function.

The graphics tablets produce signals of the X-axis and the Y-axis for the points where the user touches the tablet with a digital pen. These signals are the source of the pure digitized data. While writing a signature, it is possible to obtain signatures which differ in the case of size, orientation and deviation due to some factors (e.g. the limited space and the inclination of the tablet) even with the same person. Such differences may adversely affect the reliability of a verification system. Therefore, all signatures should be transformed in the case of position, rotation and size to decrease the differences in the signals. We follow the procedures described in Lam and Kamins (1989), Yanikoglu and Kholmatov (2009) to carry out the transformation process, defined as follows:

1. Elimination of discontinuities: It is possible to observe discontinuities (broken lines) in signals, if the user detaches the pen from the tablet while writing the signature. To recombine the broken lines in signals, two interpolation techniques can be used. If the space between broken lines is wide, the newton interpolation can be used; otherwise, the linear interpolation can be applied. For more detailed information concerning the reconstruction of broken lines, please see Samet and Hancer (2012).

2. Elimination of the signature inclination: The angles of a signature written by the same user is possible to be varied. The signature inclination is reduced using an exponential function with respect to the X-axis, defined as in Yanikoglu and Kholmatov (2009).

3. Normalization: The user can write his or her signature in any area of the tablet. Accordingly, the positions of the signature may differ. To address this issue, normalization is applied by subtracting the mean value from each value of the signal.

4. Scaling: The same person can write his or her signature in different sizes. It is therefore necessary to scale their size at a reasonable rate.

A set of dynamic signature features described by authors of Fierrez-Aguilar et al. (2005) are used in this work. One hundred features were selected. The initial data for the feature extraction are x and y signals ( x 1 , x 2 , … , x N ; y 1 , y 2 , … , y N ). Some of the features used require signals that characterize the velocity and acceleration of the digital pen. For the velocity signals, their absolute values and values for each coordinate are computed; for the acceleration signals, only their relative values are used. These signals can be obtained using the x and y signals. The absolute velocity signals at certain instants can be measured using the distance between two adjacent touch points of the pen. The absolute velocity signals are defined as follows: v i = ( x i + 1 − x i ) 2 + ( y i + 1 − y i ) 2 Δ t , where i = 1 , 2 , … , N − 1 and Δ t is the duration of an instant. The velocity signals in the respective coordinates are computed as follows: V x i = x i + 1 − x i Δ t , v y i = y i + 1 − y i Δ t , where i = 1 , 2 , … , N − 1. The relative acceleration signals in the respective coordinates are computed as follows: a x i = v x i + 1 − v x i Δ t , v y i = v y i + 1 − v y i Δ t , where i = 1 , 2 , … , N − 2.

The signature signals were taken from the SVC2004 dataset (Task 1) (Yeung et al., 2004). There were 40 sets of signatures. Each set contained 20 genuine signatures from one signer and 20 skilled forgeries from at least four other signers. Each signer was asked to provide 20 genuine signatures. For privacy reasons, the signers were advised not to use their real signatures. Instead, each of them was asked to design a new signature and practice writing it so that this new signature remained relatively consistent over different signature instances, just like a real one. The signers were also reminded that consistency should not be limited to the spatial consistency in the signature shape and it should also include the temporal consistency of the dynamic features.

Table 1 shows the designations of the studied PSO modifications. The parameter settings for these PSO modifications are presented as follows. The population size is set to 50 and the maximum number of evaluations is chosen as 2500 for all the methods. The parameters of local are set to the default values defined in Hancer (2019). The following parameters were used in methods using transformation functions: positive acceleration coefficients c 1 and c 2 are equal to 1, the coefficient of inertia w is 0.5.

Table 2 presents the results of constructing classifiers using the binary PSO variants in terms of different α values. Four characteristics are given: overall accuracy, type I error (False Rejection Rate, FRR), type II error (False Acceptance Rate, FAR), and the number of selected features (Features). Type I error shows what percentage of legitimate users was mistaken by the classifier for an intruder. Type II error demonstrates the percentage of intruders who managed to deceive the classifier and pass their signature off as the signature of a legitimate user. Bold text in the table indicates the best results within a group with one value of the coefficient α in the fitness function.

The presented results show that with a value of α less than one, the number of features in the set decreases with any method. In this case, the overall accuracy of the classification is either improving (methods standardFF, newFF, local, merge) or decreasing (methods opmerge, tanh, Vshaped). In general, the best ability to reduce features with a decrease in the coefficient α was demonstrated by the methods opmerge, local and Vshaped. It should be indicated that the classification model built on a smaller number of features can even provide a remarkable performance from the perspective of the other metrics. For instance, opmerge can achieve competitive verification results despite selecting only 2 features from 100 available features. For another instance, local shows the best verification performance for lower α cases by reducing the dimensionality at least ten times compared to the original feature set. In other words, the corresponding methods have shown good skills to detect noisy features or variables. It is therefore possible to build less complex and more generalized classification models that can obtain promising verification results.

Table 3 summarizes the mean ranks calculated according to Friedman’s criterion for different metrics and values of the coefficient α (at 40 observations and 6 degrees of freedom). The ranks for Accuracy and Errors (FAR, FRR) are multidirectional. To bring them to a single scale, we consider 1-Accuracy (called Error) instead of Accuracy. Since the p-value in all cases was equal to zero, which is less than the significance level (0.05), it can be concluded that there is a significant statistical difference in the results of the modification methods used.

Graphs have been created on the basis of the obtained average rank values in order to assess the Pareto front in terms of the ratio of metrics to the number of selected features. In the first group of graphs shown in Fig. 1, the inverse metric, classification error, was used instead of accuracy.

From the graphs obtained, it is clear that in all cases the use of the opmerge method provides the smallest set of features. With α coefficients equal to 0.7 and 0.5, PSO with the newFF fitness function allowed us to obtain a classifier with the lowest error. The best results in terms of the ratio of error and number of features were demonstrated by PSO with the local modification when the value of the coefficient α is less than one.

Fig. 2 shows plots with a Pareto front for the ratio of the type I error (FRR) and the number of selected features. The smallest value of the type I error for any value of the coefficient α was obtained using newFF, but this method coped worst of all with the selection of features.

Fig. 3 shows graphs with a Pareto front in terms of the ratio of type II error (FAR) and the number of selected features. Opmerge and local showed the lowest type II error values. For the first, this statement is valid for any value of the coefficient α, for the second, only for α less than 1.