LLVM  8.0.0svn
X86ISelLowering.h
Go to the documentation of this file.
1 //===-- X86ISelLowering.h - X86 DAG Lowering Interface ----------*- C++ -*-===//
2 //
3 // The LLVM Compiler Infrastructure
4 //
5 // This file is distributed under the University of Illinois Open Source
6 // License. See LICENSE.TXT for details.
7 //
8 //===----------------------------------------------------------------------===//
9 //
10 // This file defines the interfaces that X86 uses to lower LLVM code into a
11 // selection DAG.
12 //
13 //===----------------------------------------------------------------------===//
14 
15 #ifndef LLVM_LIB_TARGET_X86_X86ISELLOWERING_H
16 #define LLVM_LIB_TARGET_X86_X86ISELLOWERING_H
17 
22 
23 namespace llvm {
24  class X86Subtarget;
25  class X86TargetMachine;
26 
27  namespace X86ISD {
28  // X86 Specific DAG Nodes
29  enum NodeType : unsigned {
30  // Start the numbering where the builtin ops leave off.
32 
33  /// Bit scan forward.
34  BSF,
35  /// Bit scan reverse.
36  BSR,
37 
38  /// Double shift instructions. These correspond to
39  /// X86::SHLDxx and X86::SHRDxx instructions.
42 
43  /// Bitwise logical AND of floating point values. This corresponds
44  /// to X86::ANDPS or X86::ANDPD.
46 
47  /// Bitwise logical OR of floating point values. This corresponds
48  /// to X86::ORPS or X86::ORPD.
49  FOR,
50 
51  /// Bitwise logical XOR of floating point values. This corresponds
52  /// to X86::XORPS or X86::XORPD.
54 
55  /// Bitwise logical ANDNOT of floating point values. This
56  /// corresponds to X86::ANDNPS or X86::ANDNPD.
58 
59  /// These operations represent an abstract X86 call
60  /// instruction, which includes a bunch of information. In particular the
61  /// operands of these node are:
62  ///
63  /// #0 - The incoming token chain
64  /// #1 - The callee
65  /// #2 - The number of arg bytes the caller pushes on the stack.
66  /// #3 - The number of arg bytes the callee pops off the stack.
67  /// #4 - The value to pass in AL/AX/EAX (optional)
68  /// #5 - The value to pass in DL/DX/EDX (optional)
69  ///
70  /// The result values of these nodes are:
71  ///
72  /// #0 - The outgoing token chain
73  /// #1 - The first register result value (optional)
74  /// #2 - The second register result value (optional)
75  ///
77 
78  /// Same as call except it adds the NoTrack prefix.
80 
81  /// This operation implements the lowering for readcyclecounter.
83 
84  /// X86 Read Time-Stamp Counter and Processor ID.
86 
87  /// X86 Read Performance Monitoring Counters.
89 
90  /// X86 compare and logical compare instructions.
92 
93  /// X86 bit-test instructions.
94  BT,
95 
96  /// X86 SetCC. Operand 0 is condition code, and operand 1 is the EFLAGS
97  /// operand, usually produced by a CMP instruction.
99 
100  /// X86 Select
102 
103  // Same as SETCC except it's materialized with a sbb and the value is all
104  // one's or all zero's.
105  SETCC_CARRY, // R = carry_bit ? ~0 : 0
106 
107  /// X86 FP SETCC, implemented with CMP{cc}SS/CMP{cc}SD.
108  /// Operands are two FP values to compare; result is a mask of
109  /// 0s or 1s. Generally DTRT for C/C++ with NaNs.
111 
112  /// X86 FP SETCC, similar to above, but with output as an i1 mask and
113  /// with optional rounding mode.
115 
116  /// X86 conditional moves. Operand 0 and operand 1 are the two values
117  /// to select from. Operand 2 is the condition code, and operand 3 is the
118  /// flag operand produced by a CMP or TEST instruction. It also writes a
119  /// flag result.
121 
122  /// X86 conditional branches. Operand 0 is the chain operand, operand 1
123  /// is the block to branch if condition is true, operand 2 is the
124  /// condition code, and operand 3 is the flag operand produced by a CMP
125  /// or TEST instruction.
127 
128  /// BRIND node with NoTrack prefix. Operand 0 is the chain operand and
129  /// operand 1 is the target address.
131 
132  /// Return with a flag operand. Operand 0 is the chain operand, operand
133  /// 1 is the number of bytes of stack to pop.
135 
136  /// Return from interrupt. Operand 0 is the number of bytes to pop.
138 
139  /// Repeat fill, corresponds to X86::REP_STOSx.
141 
142  /// Repeat move, corresponds to X86::REP_MOVSx.
144 
145  /// On Darwin, this node represents the result of the popl
146  /// at function entry, used for PIC code.
148 
149  /// A wrapper node for TargetConstantPool, TargetJumpTable,
150  /// TargetExternalSymbol, TargetGlobalAddress, TargetGlobalTLSAddress,
151  /// MCSymbol and TargetBlockAddress.
153 
154  /// Special wrapper used under X86-64 PIC mode for RIP
155  /// relative displacements.
157 
158  /// Copies a 64-bit value from the low word of an XMM vector
159  /// to an MMX vector.
161 
162  /// Copies a 32-bit value from the low word of a MMX
163  /// vector to a GPR.
165 
166  /// Copies a GPR into the low 32-bit word of a MMX vector
167  /// and zero out the high word.
169 
170  /// Extract an 8-bit value from a vector and zero extend it to
171  /// i32, corresponds to X86::PEXTRB.
173 
174  /// Extract a 16-bit value from a vector and zero extend it to
175  /// i32, corresponds to X86::PEXTRW.
177 
178  /// Insert any element of a 4 x float vector into any element
179  /// of a destination 4 x floatvector.
181 
182  /// Insert the lower 8-bits of a 32-bit value to a vector,
183  /// corresponds to X86::PINSRB.
185 
186  /// Insert the lower 16-bits of a 32-bit value to a vector,
187  /// corresponds to X86::PINSRW.
189 
190  /// Shuffle 16 8-bit values within a vector.
192 
193  /// Compute Sum of Absolute Differences.
195  /// Compute Double Block Packed Sum-Absolute-Differences
197 
198  /// Bitwise Logical AND NOT of Packed FP values.
200 
201  /// Blend where the selector is an immediate.
203 
204  /// Dynamic (non-constant condition) vector blend where only the sign bits
205  /// of the condition elements are used. This is used to enforce that the
206  /// condition mask is not valid for generic VSELECT optimizations.
208 
209  /// Combined add and sub on an FP vector.
211 
212  // FP vector ops with rounding mode.
220 
221  // FP vector get exponent.
223  // Extract Normalized Mantissas.
225  // FP Scale.
228 
229  // Integer add/sub with unsigned saturation.
232 
233  // Integer add/sub with signed saturation.
236 
237  // Unsigned Integer average.
239 
240  /// Integer horizontal add/sub.
243 
244  /// Floating point horizontal add/sub.
247 
248  // Detect Conflicts Within a Vector
250 
251  /// Floating point max and min.
253 
254  /// Commutative FMIN and FMAX.
256 
257  /// Scalar intrinsic floating point max and min.
259 
260  /// Floating point reciprocal-sqrt and reciprocal approximation.
261  /// Note that these typically require refinement
262  /// in order to obtain suitable precision.
264 
265  // AVX-512 reciprocal approximations with a little more precision.
267 
268  // Thread Local Storage.
270 
271  // Thread Local Storage. A call to get the start address
272  // of the TLS block for the current module.
274 
275  // Thread Local Storage. When calling to an OS provided
276  // thunk at the address from an earlier relocation.
278 
279  // Exception Handling helpers.
281 
282  // SjLj exception handling setjmp.
284 
285  // SjLj exception handling longjmp.
287 
288  // SjLj exception handling dispatch.
290 
291  /// Tail call return. See X86TargetLowering::LowerCall for
292  /// the list of operands.
294 
295  // Vector move to low scalar and zero higher vector elements.
297 
298  // Vector integer zero-extend.
300  // Vector integer signed-extend.
302 
303  // Vector integer truncate.
305  // Vector integer truncate with unsigned/signed saturation.
307 
308  // Vector FP extend.
310 
311  // Vector FP round.
313 
314  // 128-bit vector logical left / right shift
316 
317  // Vector shift elements
319 
320  // Vector variable shift right arithmetic.
321  // Unlike ISD::SRA, in case shift count greater then element size
322  // use sign bit to fill destination data element.
324 
325  // Vector shift elements by immediate
327 
328  // Shifts of mask registers.
330 
331  // Bit rotate by immediate
333 
334  // Vector packed double/float comparison.
336 
337  // Vector integer comparisons.
339 
340  // v8i16 Horizontal minimum and position.
342 
344 
345  /// Vector comparison generating mask bits for fp and
346  /// integer signed and unsigned data types.
348  // Vector comparison with rounding mode for FP values
350 
351  // Arithmetic operations with FLAGS results.
353  INC, DEC, OR, XOR, AND,
354 
355  // Bit field extract.
357 
358  // Zero High Bits Starting with Specified Bit Position.
360 
361  // LOW, HI, FLAGS = umul LHS, RHS.
363 
364  // 8-bit SMUL/UMUL - AX, FLAGS = smul8/umul8 AL, RHS.
366 
367  // X86-specific multiply by immediate.
369 
370  // Vector sign bit extraction.
372 
373  // Vector bitwise comparisons.
375 
376  // Vector packed fp sign bitwise comparisons.
378 
379  // OR/AND test for masks.
382 
383  // ADD for masks.
385 
386  // Several flavors of instructions with vector shuffle behaviors.
387  // Saturated signed/unnsigned packing.
390  // Intra-lane alignr.
392  // AVX512 inter-lane alignr.
398  // VBMI2 Concat & Shift.
403  //Shuffle Packed Values at 128-bit granularity.
418 
419  // Variable Permute (VPERM).
420  // Res = VPERMV MaskV, V0
422 
423  // 3-op Variable Permute (VPERMT2).
424  // Res = VPERMV3 V0, MaskV, V1
426 
427  // Bitwise ternary logic.
429  // Fix Up Special Packed Float32/64 values.
432  // Range Restriction Calculation For Packed Pairs of Float32/64 values.
434  // Reduce - Perform Reduction Transformation on scalar\packed FP.
436  // RndScale - Round FP Values To Include A Given Number Of Fraction Bits.
437  // Also used by the legacy (V)ROUND intrinsics where we mask out the
438  // scaling part of the immediate.
440  // Tests Types Of a FP Values for packed types.
442  // Tests Types Of a FP Values for scalar types.
444 
445  // Broadcast scalar to vector.
447  // Broadcast mask to vector.
449  // Broadcast subvector to vector.
451 
452  /// SSE4A Extraction and Insertion.
454 
455  // XOP arithmetic/logical shifts.
457  // XOP signed/unsigned integer comparisons.
459  // XOP packed permute bytes.
461  // XOP two source permutation.
463 
464  // Vector multiply packed unsigned doubleword integers.
466  // Vector multiply packed signed doubleword integers.
468  // Vector Multiply Packed UnsignedIntegers with Round and Scale.
470 
471  // Multiply and Add Packed Integers.
473 
474  // AVX512IFMA multiply and add.
475  // NOTE: These are different than the instruction and perform
476  // op0 x op1 + op2.
478 
479  // VNNI
484 
485  // FMA nodes.
486  // We use the target independent ISD::FMA for the non-inverted case.
492 
493  // FMA with rounding mode.
500 
501  // Compress and expand.
504 
505  // Bits shuffle
507 
508  // Convert Unsigned/Integer to Floating-Point Value with rounding mode.
511 
512  // Vector float/double to signed/unsigned integer.
514  // Scalar float/double to signed/unsigned integer.
516 
517  // Vector float/double to signed/unsigned integer with truncation.
519  // Scalar float/double to signed/unsigned integer with truncation.
521 
522  // Vector signed/unsigned integer to float/double.
524 
525  // Save xmm argument registers to the stack, according to %al. An operator
526  // is needed so that this can be expanded with control flow.
528 
529  // Windows's _chkstk call to do stack probing.
531 
532  // For allocating variable amounts of stack space when using
533  // segmented stacks. Check if the current stacklet has enough space, and
534  // falls back to heap allocation if not.
536 
537  // Memory barriers.
540 
541  // Store FP status word into i16 register.
543 
544  // Store contents of %ah into %eflags.
546 
547  // Get a random integer and indicate whether it is valid in CF.
549 
550  // Get a NIST SP800-90B & C compliant random integer and
551  // indicate whether it is valid in CF.
553 
554  // SSE42 string comparisons.
555  // These nodes produce 3 results, index, mask, and flags. X86ISelDAGToDAG
556  // will emit one or two instructions based on which results are used. If
557  // flags and index/mask this allows us to use a single instruction since
558  // we won't have to pick and opcode for flags. Instead we can rely on the
559  // DAG to CSE everything and decide at isel.
562 
563  // Test if in transactional execution.
565 
566  // ERI instructions.
568 
569  // Conversions between float and half-float.
571 
572  // Galois Field Arithmetic Instructions
574 
575  // LWP insert record.
577 
578  // User level wait
580 
581  // Compare and swap.
587 
588  /// LOCK-prefixed arithmetic read-modify-write instructions.
589  /// EFLAGS, OUTCHAIN = LADD(INCHAIN, PTR, RHS)
591 
592  // Load, scalar_to_vector, and zero extend.
594 
595  // Store FP control world into i16 memory.
597 
598  /// This instruction implements FP_TO_SINT with the
599  /// integer destination in memory and a FP reg source. This corresponds
600  /// to the X86::FIST*m instructions and the rounding mode change stuff. It
601  /// has two inputs (token chain and address) and two outputs (int value
602  /// and token chain).
606 
607  /// This instruction implements SINT_TO_FP with the
608  /// integer source in memory and FP reg result. This corresponds to the
609  /// X86::FILD*m instructions. It has three inputs (token chain, address,
610  /// and source type) and two outputs (FP value and token chain). FILD_FLAG
611  /// also produces a flag).
614 
615  /// This instruction implements an extending load to FP stack slots.
616  /// This corresponds to the X86::FLD32m / X86::FLD64m. It takes a chain
617  /// operand, ptr to load from, and a ValueType node indicating the type
618  /// to load to.
620 
621  /// This instruction implements a truncating store to FP stack
622  /// slots. This corresponds to the X86::FST32m / X86::FST64m. It takes a
623  /// chain operand, value to store, address, and a ValueType to store it
624  /// as.
626 
627  /// This instruction grabs the address of the next argument
628  /// from a va_list. (reads and modifies the va_list in memory)
630 
631  // Vector truncating store with unsigned/signed saturation
633  // Vector truncating masked store with unsigned/signed saturation
635 
636  // X86 specific gather and scatter
638 
639  // WARNING: Do not add anything in the end unless you want the node to
640  // have memop! In fact, starting from FIRST_TARGET_MEMORY_OPCODE all
641  // opcodes will be thought as target memory ops!
642  };
643  } // end namespace X86ISD
644 
645  /// Define some predicates that are used for node matching.
646  namespace X86 {
647  /// Returns true if Elt is a constant zero or floating point constant +0.0.
648  bool isZeroNode(SDValue Elt);
649 
650  /// Returns true of the given offset can be
651  /// fit into displacement field of the instruction.
653  bool hasSymbolicDisplacement = true);
654 
655  /// Determines whether the callee is required to pop its
656  /// own arguments. Callee pop is necessary to support tail calls.
658  bool is64Bit, bool IsVarArg, bool GuaranteeTCO);
659 
660  } // end namespace X86
661 
662  //===--------------------------------------------------------------------===//
663  // X86 Implementation of the TargetLowering interface
664  class X86TargetLowering final : public TargetLowering {
665  public:
666  explicit X86TargetLowering(const X86TargetMachine &TM,
667  const X86Subtarget &STI);
668 
669  unsigned getJumpTableEncoding() const override;
670  bool useSoftFloat() const override;
671 
672  void markLibCallAttributes(MachineFunction *MF, unsigned CC,
673  ArgListTy &Args) const override;
674 
675  MVT getScalarShiftAmountTy(const DataLayout &, EVT VT) const override {
676  return MVT::i8;
677  }
678 
679  const MCExpr *
680  LowerCustomJumpTableEntry(const MachineJumpTableInfo *MJTI,
681  const MachineBasicBlock *MBB, unsigned uid,
682  MCContext &Ctx) const override;
683 
684  /// Returns relocation base for the given PIC jumptable.
685  SDValue getPICJumpTableRelocBase(SDValue Table,
686  SelectionDAG &DAG) const override;
687  const MCExpr *
688  getPICJumpTableRelocBaseExpr(const MachineFunction *MF,
689  unsigned JTI, MCContext &Ctx) const override;
690 
691  /// Return the desired alignment for ByVal aggregate
692  /// function arguments in the caller parameter area. For X86, aggregates
693  /// that contains are placed at 16-byte boundaries while the rest are at
694  /// 4-byte boundaries.
695  unsigned getByValTypeAlignment(Type *Ty,
696  const DataLayout &DL) const override;
697 
698  /// Returns the target specific optimal type for load
699  /// and store operations as a result of memset, memcpy, and memmove
700  /// lowering. If DstAlign is zero that means it's safe to destination
701  /// alignment can satisfy any constraint. Similarly if SrcAlign is zero it
702  /// means there isn't a need to check it against alignment requirement,
703  /// probably because the source does not need to be loaded. If 'IsMemset' is
704  /// true, that means it's expanding a memset. If 'ZeroMemset' is true, that
705  /// means it's a memset of zero. 'MemcpyStrSrc' indicates whether the memcpy
706  /// source is constant so it does not need to be loaded.
707  /// It returns EVT::Other if the type should be determined using generic
708  /// target-independent logic.
709  EVT getOptimalMemOpType(uint64_t Size, unsigned DstAlign, unsigned SrcAlign,
710  bool IsMemset, bool ZeroMemset, bool MemcpyStrSrc,
711  MachineFunction &MF) const override;
712 
713  /// Returns true if it's safe to use load / store of the
714  /// specified type to expand memcpy / memset inline. This is mostly true
715  /// for all types except for some special cases. For example, on X86
716  /// targets without SSE2 f64 load / store are done with fldl / fstpl which
717  /// also does type conversion. Note the specified type doesn't have to be
718  /// legal as the hook is used before type legalization.
719  bool isSafeMemOpType(MVT VT) const override;
720 
721  /// Returns true if the target allows unaligned memory accesses of the
722  /// specified type. Returns whether it is "fast" in the last argument.
723  bool allowsMisalignedMemoryAccesses(EVT VT, unsigned AS, unsigned Align,
724  bool *Fast) const override;
725 
726  /// Provide custom lowering hooks for some operations.
727  ///
728  SDValue LowerOperation(SDValue Op, SelectionDAG &DAG) const override;
729 
730  /// Places new result values for the node in Results (their number
731  /// and types must exactly match those of the original return values of
732  /// the node), or leaves Results empty, which indicates that the node is not
733  /// to be custom lowered after all.
734  void LowerOperationWrapper(SDNode *N,
736  SelectionDAG &DAG) const override;
737 
738  /// Replace the results of node with an illegal result
739  /// type with new values built out of custom code.
740  ///
741  void ReplaceNodeResults(SDNode *N, SmallVectorImpl<SDValue>&Results,
742  SelectionDAG &DAG) const override;
743 
744  SDValue PerformDAGCombine(SDNode *N, DAGCombinerInfo &DCI) const override;
745 
746  // Return true if it is profitable to combine a BUILD_VECTOR with a
747  // stride-pattern to a shuffle and a truncate.
748  // Example of such a combine:
749  // v4i32 build_vector((extract_elt V, 1),
750  // (extract_elt V, 3),
751  // (extract_elt V, 5),
752  // (extract_elt V, 7))
753  // -->
754  // v4i32 truncate (bitcast (shuffle<1,u,3,u,4,u,5,u,6,u,7,u> V, u) to
755  // v4i64)
756  bool isDesirableToCombineBuildVectorToShuffleTruncate(
757  ArrayRef<int> ShuffleMask, EVT SrcVT, EVT TruncVT) const override;
758 
759  /// Return true if the target has native support for
760  /// the specified value type and it is 'desirable' to use the type for the
761  /// given node type. e.g. On x86 i16 is legal, but undesirable since i16
762  /// instruction encodings are longer and some i16 instructions are slow.
763  bool isTypeDesirableForOp(unsigned Opc, EVT VT) const override;
764 
765  /// Return true if the target has native support for the
766  /// specified value type and it is 'desirable' to use the type. e.g. On x86
767  /// i16 is legal, but undesirable since i16 instruction encodings are longer
768  /// and some i16 instructions are slow.
769  bool IsDesirableToPromoteOp(SDValue Op, EVT &PVT) const override;
770 
772  EmitInstrWithCustomInserter(MachineInstr &MI,
773  MachineBasicBlock *MBB) const override;
774 
775  /// This method returns the name of a target specific DAG node.
776  const char *getTargetNodeName(unsigned Opcode) const override;
777 
778  bool mergeStoresAfterLegalization() const override { return true; }
779 
780  bool canMergeStoresTo(unsigned AddressSpace, EVT MemVT,
781  const SelectionDAG &DAG) const override;
782 
783  bool isCheapToSpeculateCttz() const override;
784 
785  bool isCheapToSpeculateCtlz() const override;
786 
787  bool isCtlzFast() const override;
788 
789  bool hasBitPreservingFPLogic(EVT VT) const override {
790  return VT == MVT::f32 || VT == MVT::f64 || VT.isVector();
791  }
792 
793  bool isMultiStoresCheaperThanBitsMerge(EVT LTy, EVT HTy) const override {
794  // If the pair to store is a mixture of float and int values, we will
795  // save two bitwise instructions and one float-to-int instruction and
796  // increase one store instruction. There is potentially a more
797  // significant benefit because it avoids the float->int domain switch
798  // for input value. So It is more likely a win.
799  if ((LTy.isFloatingPoint() && HTy.isInteger()) ||
800  (LTy.isInteger() && HTy.isFloatingPoint()))
801  return true;
802  // If the pair only contains int values, we will save two bitwise
803  // instructions and increase one store instruction (costing one more
804  // store buffer). Since the benefit is more blurred so we leave
805  // such pair out until we get testcase to prove it is a win.
806  return false;
807  }
808 
809  bool isMaskAndCmp0FoldingBeneficial(const Instruction &AndI) const override;
810 
811  bool hasAndNotCompare(SDValue Y) const override;
812 
813  bool hasAndNot(SDValue Y) const override;
814 
815  bool preferShiftsToClearExtremeBits(SDValue Y) const override;
816 
817  bool
819  unsigned KeptBits) const override {
820  // For vectors, we don't have a preference..
821  if (XVT.isVector())
822  return false;
823 
824  auto VTIsOk = [](EVT VT) -> bool {
825  return VT == MVT::i8 || VT == MVT::i16 || VT == MVT::i32 ||
826  VT == MVT::i64;
827  };
828 
829  // We are ok with KeptBitsVT being byte/word/dword, what MOVS supports.
830  // XVT will be larger than KeptBitsVT.
831  MVT KeptBitsVT = MVT::getIntegerVT(KeptBits);
832  return VTIsOk(XVT) && VTIsOk(KeptBitsVT);
833  }
834 
835  bool shouldSplatInsEltVarIndex(EVT VT) const override;
836 
837  bool convertSetCCLogicToBitwiseLogic(EVT VT) const override {
838  return VT.isScalarInteger();
839  }
840 
841  /// Vector-sized comparisons are fast using PCMPEQ + PMOVMSK or PTEST.
842  MVT hasFastEqualityCompare(unsigned NumBits) const override;
843 
844  /// Allow multiple load pairs per block for smaller and faster code.
845  unsigned getMemcmpEqZeroLoadsPerBlock() const override {
846  return 2;
847  }
848 
849  /// Return the value type to use for ISD::SETCC.
850  EVT getSetCCResultType(const DataLayout &DL, LLVMContext &Context,
851  EVT VT) const override;
852 
853  bool targetShrinkDemandedConstant(SDValue Op, const APInt &Demanded,
854  TargetLoweringOpt &TLO) const override;
855 
856  /// Determine which of the bits specified in Mask are known to be either
857  /// zero or one and return them in the KnownZero/KnownOne bitsets.
858  void computeKnownBitsForTargetNode(const SDValue Op,
859  KnownBits &Known,
860  const APInt &DemandedElts,
861  const SelectionDAG &DAG,
862  unsigned Depth = 0) const override;
863 
864  /// Determine the number of bits in the operation that are sign bits.
865  unsigned ComputeNumSignBitsForTargetNode(SDValue Op,
866  const APInt &DemandedElts,
867  const SelectionDAG &DAG,
868  unsigned Depth) const override;
869 
870  bool SimplifyDemandedVectorEltsForTargetNode(SDValue Op,
871  const APInt &DemandedElts,
872  APInt &KnownUndef,
873  APInt &KnownZero,
874  TargetLoweringOpt &TLO,
875  unsigned Depth) const override;
876 
877  SDValue unwrapAddress(SDValue N) const override;
878 
879  bool isGAPlusOffset(SDNode *N, const GlobalValue* &GA,
880  int64_t &Offset) const override;
881 
882  SDValue getReturnAddressFrameIndex(SelectionDAG &DAG) const;
883 
884  bool ExpandInlineAsm(CallInst *CI) const override;
885 
886  ConstraintType getConstraintType(StringRef Constraint) const override;
887 
888  /// Examine constraint string and operand type and determine a weight value.
889  /// The operand object must already have been set up with the operand type.
891  getSingleConstraintMatchWeight(AsmOperandInfo &info,
892  const char *constraint) const override;
893 
894  const char *LowerXConstraint(EVT ConstraintVT) const override;
895 
896  /// Lower the specified operand into the Ops vector. If it is invalid, don't
897  /// add anything to Ops. If hasMemory is true it means one of the asm
898  /// constraint of the inline asm instruction being processed is 'm'.
899  void LowerAsmOperandForConstraint(SDValue Op,
900  std::string &Constraint,
901  std::vector<SDValue> &Ops,
902  SelectionDAG &DAG) const override;
903 
904  unsigned
905  getInlineAsmMemConstraint(StringRef ConstraintCode) const override {
906  if (ConstraintCode == "i")
908  else if (ConstraintCode == "o")
910  else if (ConstraintCode == "v")
912  else if (ConstraintCode == "X")
914  return TargetLowering::getInlineAsmMemConstraint(ConstraintCode);
915  }
916 
917  /// Given a physical register constraint
918  /// (e.g. {edx}), return the register number and the register class for the
919  /// register. This should only be used for C_Register constraints. On
920  /// error, this returns a register number of 0.
921  std::pair<unsigned, const TargetRegisterClass *>
922  getRegForInlineAsmConstraint(const TargetRegisterInfo *TRI,
923  StringRef Constraint, MVT VT) const override;
924 
925  /// Return true if the addressing mode represented
926  /// by AM is legal for this target, for a load/store of the specified type.
927  bool isLegalAddressingMode(const DataLayout &DL, const AddrMode &AM,
928  Type *Ty, unsigned AS,
929  Instruction *I = nullptr) const override;
930 
931  /// Return true if the specified immediate is legal
932  /// icmp immediate, that is the target has icmp instructions which can
933  /// compare a register against the immediate without having to materialize
934  /// the immediate into a register.
935  bool isLegalICmpImmediate(int64_t Imm) const override;
936 
937  /// Return true if the specified immediate is legal
938  /// add immediate, that is the target has add instructions which can
939  /// add a register and the immediate without having to materialize
940  /// the immediate into a register.
941  bool isLegalAddImmediate(int64_t Imm) const override;
942 
943  /// Return the cost of the scaling factor used in the addressing
944  /// mode represented by AM for this target, for a load/store
945  /// of the specified type.
946  /// If the AM is supported, the return value must be >= 0.
947  /// If the AM is not supported, it returns a negative value.
948  int getScalingFactorCost(const DataLayout &DL, const AddrMode &AM, Type *Ty,
949  unsigned AS) const override;
950 
951  bool isVectorShiftByScalarCheap(Type *Ty) const override;
952 
953  /// Return true if it's free to truncate a value of
954  /// type Ty1 to type Ty2. e.g. On x86 it's free to truncate a i32 value in
955  /// register EAX to i16 by referencing its sub-register AX.
956  bool isTruncateFree(Type *Ty1, Type *Ty2) const override;
957  bool isTruncateFree(EVT VT1, EVT VT2) const override;
958 
959  bool allowTruncateForTailCall(Type *Ty1, Type *Ty2) const override;
960 
961  /// Return true if any actual instruction that defines a
962  /// value of type Ty1 implicit zero-extends the value to Ty2 in the result
963  /// register. This does not necessarily include registers defined in
964  /// unknown ways, such as incoming arguments, or copies from unknown
965  /// virtual registers. Also, if isTruncateFree(Ty2, Ty1) is true, this
966  /// does not necessarily apply to truncate instructions. e.g. on x86-64,
967  /// all instructions that define 32-bit values implicit zero-extend the
968  /// result out to 64 bits.
969  bool isZExtFree(Type *Ty1, Type *Ty2) const override;
970  bool isZExtFree(EVT VT1, EVT VT2) const override;
971  bool isZExtFree(SDValue Val, EVT VT2) const override;
972 
973  /// Return true if folding a vector load into ExtVal (a sign, zero, or any
974  /// extend node) is profitable.
975  bool isVectorLoadExtDesirable(SDValue) const override;
976 
977  /// Return true if an FMA operation is faster than a pair of fmul and fadd
978  /// instructions. fmuladd intrinsics will be expanded to FMAs when this
979  /// method returns true, otherwise fmuladd is expanded to fmul + fadd.
980  bool isFMAFasterThanFMulAndFAdd(EVT VT) const override;
981 
982  /// Return true if it's profitable to narrow
983  /// operations of type VT1 to VT2. e.g. on x86, it's profitable to narrow
984  /// from i32 to i8 but not from i32 to i16.
985  bool isNarrowingProfitable(EVT VT1, EVT VT2) const override;
986 
987  /// Given an intrinsic, checks if on the target the intrinsic will need to map
988  /// to a MemIntrinsicNode (touches memory). If this is the case, it returns
989  /// true and stores the intrinsic information into the IntrinsicInfo that was
990  /// passed to the function.
991  bool getTgtMemIntrinsic(IntrinsicInfo &Info, const CallInst &I,
992  MachineFunction &MF,
993  unsigned Intrinsic) const override;
994 
995  /// Returns true if the target can instruction select the
996  /// specified FP immediate natively. If false, the legalizer will
997  /// materialize the FP immediate as a load from a constant pool.
998  bool isFPImmLegal(const APFloat &Imm, EVT VT) const override;
999 
1000  /// Targets can use this to indicate that they only support *some*
1001  /// VECTOR_SHUFFLE operations, those with specific masks. By default, if a
1002  /// target supports the VECTOR_SHUFFLE node, all mask values are assumed to
1003  /// be legal.
1004  bool isShuffleMaskLegal(ArrayRef<int> Mask, EVT VT) const override;
1005 
1006  /// Similar to isShuffleMaskLegal. Targets can use this to indicate if there
1007  /// is a suitable VECTOR_SHUFFLE that can be used to replace a VAND with a
1008  /// constant pool entry.
1009  bool isVectorClearMaskLegal(ArrayRef<int> Mask, EVT VT) const override;
1010 
1011  /// Returns true if lowering to a jump table is allowed.
1012  bool areJTsAllowed(const Function *Fn) const override;
1013 
1014  /// If true, then instruction selection should
1015  /// seek to shrink the FP constant of the specified type to a smaller type
1016  /// in order to save space and / or reduce runtime.
1017  bool ShouldShrinkFPConstant(EVT VT) const override {
1018  // Don't shrink FP constpool if SSE2 is available since cvtss2sd is more
1019  // expensive than a straight movsd. On the other hand, it's important to
1020  // shrink long double fp constant since fldt is very slow.
1021  return !X86ScalarSSEf64 || VT == MVT::f80;
1022  }
1023 
1024  /// Return true if we believe it is correct and profitable to reduce the
1025  /// load node to a smaller type.
1026  bool shouldReduceLoadWidth(SDNode *Load, ISD::LoadExtType ExtTy,
1027  EVT NewVT) const override;
1028 
1029  /// Return true if the specified scalar FP type is computed in an SSE
1030  /// register, not on the X87 floating point stack.
1031  bool isScalarFPTypeInSSEReg(EVT VT) const {
1032  return (VT == MVT::f64 && X86ScalarSSEf64) || // f64 is when SSE2
1033  (VT == MVT::f32 && X86ScalarSSEf32); // f32 is when SSE1
1034  }
1035 
1036  /// Returns true if it is beneficial to convert a load of a constant
1037  /// to just the constant itself.
1038  bool shouldConvertConstantLoadToIntImm(const APInt &Imm,
1039  Type *Ty) const override;
1040 
1041  bool convertSelectOfConstantsToMath(EVT VT) const override;
1042 
1043  bool decomposeMulByConstant(EVT VT, SDValue C) const override;
1044 
1045  /// Return true if EXTRACT_SUBVECTOR is cheap for this result type
1046  /// with this index.
1047  bool isExtractSubvectorCheap(EVT ResVT, EVT SrcVT,
1048  unsigned Index) const override;
1049 
1050  bool storeOfVectorConstantIsCheap(EVT MemVT, unsigned NumElem,
1051  unsigned AddrSpace) const override {
1052  // If we can replace more than 2 scalar stores, there will be a reduction
1053  // in instructions even after we add a vector constant load.
1054  return NumElem > 2;
1055  }
1056 
1057  bool isLoadBitCastBeneficial(EVT LoadVT, EVT BitcastVT) const override;
1058 
1059  /// Intel processors have a unified instruction and data cache
1060  const char * getClearCacheBuiltinName() const override {
1061  return nullptr; // nothing to do, move along.
1062  }
1063 
1064  unsigned getRegisterByName(const char* RegName, EVT VT,
1065  SelectionDAG &DAG) const override;
1066 
1067  /// If a physical register, this returns the register that receives the
1068  /// exception address on entry to an EH pad.
1069  unsigned
1070  getExceptionPointerRegister(const Constant *PersonalityFn) const override;
1071 
1072  /// If a physical register, this returns the register that receives the
1073  /// exception typeid on entry to a landing pad.
1074  unsigned
1075  getExceptionSelectorRegister(const Constant *PersonalityFn) const override;
1076 
1077  virtual bool needsFixedCatchObjects() const override;
1078 
1079  /// This method returns a target specific FastISel object,
1080  /// or null if the target does not support "fast" ISel.
1082  const TargetLibraryInfo *libInfo) const override;
1083 
1084  /// If the target has a standard location for the stack protector cookie,
1085  /// returns the address of that location. Otherwise, returns nullptr.
1086  Value *getIRStackGuard(IRBuilder<> &IRB) const override;
1087 
1088  bool useLoadStackGuardNode() const override;
1089  bool useStackGuardXorFP() const override;
1090  void insertSSPDeclarations(Module &M) const override;
1091  Value *getSDagStackGuard(const Module &M) const override;
1092  Value *getSSPStackGuardCheck(const Module &M) const override;
1093  SDValue emitStackGuardXorFP(SelectionDAG &DAG, SDValue Val,
1094  const SDLoc &DL) const override;
1095 
1096 
1097  /// Return true if the target stores SafeStack pointer at a fixed offset in
1098  /// some non-standard address space, and populates the address space and
1099  /// offset as appropriate.
1100  Value *getSafeStackPointerLocation(IRBuilder<> &IRB) const override;
1101 
1102  SDValue BuildFILD(SDValue Op, EVT SrcVT, SDValue Chain, SDValue StackSlot,
1103  SelectionDAG &DAG) const;
1104 
1105  bool isNoopAddrSpaceCast(unsigned SrcAS, unsigned DestAS) const override;
1106 
1107  /// Customize the preferred legalization strategy for certain types.
1108  LegalizeTypeAction getPreferredVectorAction(EVT VT) const override;
1109 
1110  MVT getRegisterTypeForCallingConv(LLVMContext &Context, CallingConv::ID CC,
1111  EVT VT) const override;
1112 
1113  unsigned getNumRegistersForCallingConv(LLVMContext &Context,
1114  CallingConv::ID CC,
1115  EVT VT) const override;
1116 
1117  bool isIntDivCheap(EVT VT, AttributeList Attr) const override;
1118 
1119  bool supportSwiftError() const override;
1120 
1121  StringRef getStackProbeSymbolName(MachineFunction &MF) const override;
1122 
1123  bool hasVectorBlend() const override { return true; }
1124 
1125  unsigned getMaxSupportedInterleaveFactor() const override { return 4; }
1126 
1127  /// Lower interleaved load(s) into target specific
1128  /// instructions/intrinsics.
1129  bool lowerInterleavedLoad(LoadInst *LI,
1131  ArrayRef<unsigned> Indices,
1132  unsigned Factor) const override;
1133 
1134  /// Lower interleaved store(s) into target specific
1135  /// instructions/intrinsics.
1136  bool lowerInterleavedStore(StoreInst *SI, ShuffleVectorInst *SVI,
1137  unsigned Factor) const override;
1138 
1139  SDValue expandIndirectJTBranch(const SDLoc& dl, SDValue Value,
1140  SDValue Addr, SelectionDAG &DAG)
1141  const override;
1142 
1143  protected:
1144  std::pair<const TargetRegisterClass *, uint8_t>
1145  findRepresentativeClass(const TargetRegisterInfo *TRI,
1146  MVT VT) const override;
1147 
1148  private:
1149  /// Keep a reference to the X86Subtarget around so that we can
1150  /// make the right decision when generating code for different targets.
1151  const X86Subtarget &Subtarget;
1152 
1153  /// Select between SSE or x87 floating point ops.
1154  /// When SSE is available, use it for f32 operations.
1155  /// When SSE2 is available, use it for f64 operations.
1156  bool X86ScalarSSEf32;
1157  bool X86ScalarSSEf64;
1158 
1159  /// A list of legal FP immediates.
1160  std::vector<APFloat> LegalFPImmediates;
1161 
1162  /// Indicate that this x86 target can instruction
1163  /// select the specified FP immediate natively.
1164  void addLegalFPImmediate(const APFloat& Imm) {
1165  LegalFPImmediates.push_back(Imm);
1166  }
1167 
1168  SDValue LowerCallResult(SDValue Chain, SDValue InFlag,
1169  CallingConv::ID CallConv, bool isVarArg,
1171  const SDLoc &dl, SelectionDAG &DAG,
1172  SmallVectorImpl<SDValue> &InVals,
1173  uint32_t *RegMask) const;
1174  SDValue LowerMemArgument(SDValue Chain, CallingConv::ID CallConv,
1175  const SmallVectorImpl<ISD::InputArg> &ArgInfo,
1176  const SDLoc &dl, SelectionDAG &DAG,
1177  const CCValAssign &VA, MachineFrameInfo &MFI,
1178  unsigned i) const;
1180  const SDLoc &dl, SelectionDAG &DAG,
1181  const CCValAssign &VA,
1182  ISD::ArgFlagsTy Flags) const;
1183 
1184  // Call lowering helpers.
1185 
1186  /// Check whether the call is eligible for tail call optimization. Targets
1187  /// that want to do tail call optimization should implement this function.
1188  bool IsEligibleForTailCallOptimization(SDValue Callee,
1189  CallingConv::ID CalleeCC,
1190  bool isVarArg,
1191  bool isCalleeStructRet,
1192  bool isCallerStructRet,
1193  Type *RetTy,
1194  const SmallVectorImpl<ISD::OutputArg> &Outs,
1195  const SmallVectorImpl<SDValue> &OutVals,
1196  const SmallVectorImpl<ISD::InputArg> &Ins,
1197  SelectionDAG& DAG) const;
1198  SDValue EmitTailCallLoadRetAddr(SelectionDAG &DAG, SDValue &OutRetAddr,
1199  SDValue Chain, bool IsTailCall,
1200  bool Is64Bit, int FPDiff,
1201  const SDLoc &dl) const;
1202 
1203  unsigned GetAlignedArgumentStackSize(unsigned StackSize,
1204  SelectionDAG &DAG) const;
1205 
1206  unsigned getAddressSpace(void) const;
1207 
1208  std::pair<SDValue,SDValue> FP_TO_INTHelper(SDValue Op, SelectionDAG &DAG,
1209  bool isSigned,
1210  bool isReplace) const;
1211 
1212  SDValue LowerBUILD_VECTOR(SDValue Op, SelectionDAG &DAG) const;
1213  SDValue LowerVSELECT(SDValue Op, SelectionDAG &DAG) const;
1216 
1217  unsigned getGlobalWrapperKind(const GlobalValue *GV = nullptr,
1218  const unsigned char OpFlags = 0) const;
1219  SDValue LowerConstantPool(SDValue Op, SelectionDAG &DAG) const;
1220  SDValue LowerBlockAddress(SDValue Op, SelectionDAG &DAG) const;
1221  SDValue LowerGlobalAddress(const GlobalValue *GV, const SDLoc &dl,
1222  int64_t Offset, SelectionDAG &DAG) const;
1223  SDValue LowerGlobalAddress(SDValue Op, SelectionDAG &DAG) const;
1224  SDValue LowerGlobalTLSAddress(SDValue Op, SelectionDAG &DAG) const;
1225  SDValue LowerExternalSymbol(SDValue Op, SelectionDAG &DAG) const;
1226 
1227  SDValue LowerSINT_TO_FP(SDValue Op, SelectionDAG &DAG) const;
1228  SDValue LowerUINT_TO_FP(SDValue Op, SelectionDAG &DAG) const;
1229  SDValue LowerTRUNCATE(SDValue Op, SelectionDAG &DAG) const;
1230  SDValue LowerFP_TO_INT(SDValue Op, SelectionDAG &DAG) const;
1231  SDValue LowerSETCC(SDValue Op, SelectionDAG &DAG) const;
1232  SDValue LowerSETCCCARRY(SDValue Op, SelectionDAG &DAG) const;
1233  SDValue LowerSELECT(SDValue Op, SelectionDAG &DAG) const;
1234  SDValue LowerBRCOND(SDValue Op, SelectionDAG &DAG) const;
1235  SDValue LowerJumpTable(SDValue Op, SelectionDAG &DAG) const;
1237  SDValue LowerVASTART(SDValue Op, SelectionDAG &DAG) const;
1238  SDValue LowerVAARG(SDValue Op, SelectionDAG &DAG) const;
1239  SDValue LowerRETURNADDR(SDValue Op, SelectionDAG &DAG) const;
1240  SDValue LowerADDROFRETURNADDR(SDValue Op, SelectionDAG &DAG) const;
1241  SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG) const;
1242  SDValue LowerFRAME_TO_ARGS_OFFSET(SDValue Op, SelectionDAG &DAG) const;
1243  SDValue LowerEH_RETURN(SDValue Op, SelectionDAG &DAG) const;
1244  SDValue lowerEH_SJLJ_SETJMP(SDValue Op, SelectionDAG &DAG) const;
1245  SDValue lowerEH_SJLJ_LONGJMP(SDValue Op, SelectionDAG &DAG) const;
1246  SDValue lowerEH_SJLJ_SETUP_DISPATCH(SDValue Op, SelectionDAG &DAG) const;
1247  SDValue LowerINIT_TRAMPOLINE(SDValue Op, SelectionDAG &DAG) const;
1248  SDValue LowerFLT_ROUNDS_(SDValue Op, SelectionDAG &DAG) const;
1249  SDValue LowerWin64_i128OP(SDValue Op, SelectionDAG &DAG) const;
1250  SDValue LowerGC_TRANSITION_START(SDValue Op, SelectionDAG &DAG) const;
1251  SDValue LowerGC_TRANSITION_END(SDValue Op, SelectionDAG &DAG) const;
1252  SDValue LowerINTRINSIC_WO_CHAIN(SDValue Op, SelectionDAG &DAG) const;
1253 
1254  SDValue
1255  LowerFormalArguments(SDValue Chain, CallingConv::ID CallConv, bool isVarArg,
1256  const SmallVectorImpl<ISD::InputArg> &Ins,
1257  const SDLoc &dl, SelectionDAG &DAG,
1258  SmallVectorImpl<SDValue> &InVals) const override;
1259  SDValue LowerCall(CallLoweringInfo &CLI,
1260  SmallVectorImpl<SDValue> &InVals) const override;
1261 
1262  SDValue LowerReturn(SDValue Chain, CallingConv::ID CallConv, bool isVarArg,
1263  const SmallVectorImpl<ISD::OutputArg> &Outs,
1264  const SmallVectorImpl<SDValue> &OutVals,
1265  const SDLoc &dl, SelectionDAG &DAG) const override;
1266 
1267  bool supportSplitCSR(MachineFunction *MF) const override {
1269  MF->getFunction().hasFnAttribute(Attribute::NoUnwind);
1270  }
1271  void initializeSplitCSR(MachineBasicBlock *Entry) const override;
1272  void insertCopiesSplitCSR(
1273  MachineBasicBlock *Entry,
1274  const SmallVectorImpl<MachineBasicBlock *> &Exits) const override;
1275 
1276  bool isUsedByReturnOnly(SDNode *N, SDValue &Chain) const override;
1277 
1278  bool mayBeEmittedAsTailCall(const CallInst *CI) const override;
1279 
1280  EVT getTypeForExtReturn(LLVMContext &Context, EVT VT,
1281  ISD::NodeType ExtendKind) const override;
1282 
1283  bool CanLowerReturn(CallingConv::ID CallConv, MachineFunction &MF,
1284  bool isVarArg,
1285  const SmallVectorImpl<ISD::OutputArg> &Outs,
1286  LLVMContext &Context) const override;
1287 
1288  const MCPhysReg *getScratchRegisters(CallingConv::ID CC) const override;
1289 
1291  shouldExpandAtomicLoadInIR(LoadInst *SI) const override;
1292  bool shouldExpandAtomicStoreInIR(StoreInst *SI) const override;
1294  shouldExpandAtomicRMWInIR(AtomicRMWInst *AI) const override;
1295 
1296  LoadInst *
1297  lowerIdempotentRMWIntoFencedLoad(AtomicRMWInst *AI) const override;
1298 
1299  bool needsCmpXchgNb(Type *MemType) const;
1300 
1301  void SetupEntryBlockForSjLj(MachineInstr &MI, MachineBasicBlock *MBB,
1302  MachineBasicBlock *DispatchBB, int FI) const;
1303 
1304  // Utility function to emit the low-level va_arg code for X86-64.
1306  EmitVAARG64WithCustomInserter(MachineInstr &MI,
1307  MachineBasicBlock *MBB) const;
1308 
1309  /// Utility function to emit the xmm reg save portion of va_start.
1311  EmitVAStartSaveXMMRegsWithCustomInserter(MachineInstr &BInstr,
1312  MachineBasicBlock *BB) const;
1313 
1314  MachineBasicBlock *EmitLoweredCascadedSelect(MachineInstr &MI1,
1315  MachineInstr &MI2,
1316  MachineBasicBlock *BB) const;
1317 
1318  MachineBasicBlock *EmitLoweredSelect(MachineInstr &I,
1319  MachineBasicBlock *BB) const;
1320 
1321  MachineBasicBlock *EmitLoweredAtomicFP(MachineInstr &I,
1322  MachineBasicBlock *BB) const;
1323 
1324  MachineBasicBlock *EmitLoweredCatchRet(MachineInstr &MI,
1325  MachineBasicBlock *BB) const;
1326 
1327  MachineBasicBlock *EmitLoweredCatchPad(MachineInstr &MI,
1328  MachineBasicBlock *BB) const;
1329 
1330  MachineBasicBlock *EmitLoweredSegAlloca(MachineInstr &MI,
1331  MachineBasicBlock *BB) const;
1332 
1333  MachineBasicBlock *EmitLoweredTLSAddr(MachineInstr &MI,
1334  MachineBasicBlock *BB) const;
1335 
1336  MachineBasicBlock *EmitLoweredTLSCall(MachineInstr &MI,
1337  MachineBasicBlock *BB) const;
1338 
1339  MachineBasicBlock *EmitLoweredRetpoline(MachineInstr &MI,
1340  MachineBasicBlock *BB) const;
1341 
1342  MachineBasicBlock *emitEHSjLjSetJmp(MachineInstr &MI,
1343  MachineBasicBlock *MBB) const;
1344 
1345  void emitSetJmpShadowStackFix(MachineInstr &MI,
1346  MachineBasicBlock *MBB) const;
1347 
1348  MachineBasicBlock *emitEHSjLjLongJmp(MachineInstr &MI,
1349  MachineBasicBlock *MBB) const;
1350 
1351  MachineBasicBlock *emitLongJmpShadowStackFix(MachineInstr &MI,
1352  MachineBasicBlock *MBB) const;
1353 
1354  MachineBasicBlock *emitFMA3Instr(MachineInstr &MI,
1355  MachineBasicBlock *MBB) const;
1356 
1357  MachineBasicBlock *EmitSjLjDispatchBlock(MachineInstr &MI,
1358  MachineBasicBlock *MBB) const;
1359 
1360  /// Emit nodes that will be selected as "test Op0,Op0", or something
1361  /// equivalent, for use with the given x86 condition code.
1362  SDValue EmitTest(SDValue Op0, unsigned X86CC, const SDLoc &dl,
1363  SelectionDAG &DAG) const;
1364 
1365  /// Emit nodes that will be selected as "cmp Op0,Op1", or something
1366  /// equivalent, for use with the given x86 condition code.
1367  SDValue EmitCmp(SDValue Op0, SDValue Op1, unsigned X86CC, const SDLoc &dl,
1368  SelectionDAG &DAG) const;
1369 
1370  /// Convert a comparison if required by the subtarget.
1371  SDValue ConvertCmpIfNecessary(SDValue Cmp, SelectionDAG &DAG) const;
1372 
1373  /// Check if replacement of SQRT with RSQRT should be disabled.
1374  bool isFsqrtCheap(SDValue Operand, SelectionDAG &DAG) const override;
1375 
1376  /// Use rsqrt* to speed up sqrt calculations.
1377  SDValue getSqrtEstimate(SDValue Operand, SelectionDAG &DAG, int Enabled,
1378  int &RefinementSteps, bool &UseOneConstNR,
1379  bool Reciprocal) const override;
1380 
1381  /// Use rcp* to speed up fdiv calculations.
1382  SDValue getRecipEstimate(SDValue Operand, SelectionDAG &DAG, int Enabled,
1383  int &RefinementSteps) const override;
1384 
1385  /// Reassociate floating point divisions into multiply by reciprocal.
1386  unsigned combineRepeatedFPDivisors() const override;
1387  };
1388 
1389  namespace X86 {
1391  const TargetLibraryInfo *libInfo);
1392  } // end namespace X86
1393 
1394  // Base class for all X86 non-masked store operations.
1395  class X86StoreSDNode : public MemSDNode {
1396  public:
1397  X86StoreSDNode(unsigned Opcode, unsigned Order, const DebugLoc &dl,
1398  SDVTList VTs, EVT MemVT,
1399  MachineMemOperand *MMO)
1400  :MemSDNode(Opcode, Order, dl, VTs, MemVT, MMO) {}
1401  const SDValue &getValue() const { return getOperand(1); }
1402  const SDValue &getBasePtr() const { return getOperand(2); }
1403 
1404  static bool classof(const SDNode *N) {
1405  return N->getOpcode() == X86ISD::VTRUNCSTORES ||
1407  }
1408  };
1409 
1410  // Base class for all X86 masked store operations.
1411  // The class has the same order of operands as MaskedStoreSDNode for
1412  // convenience.
1414  public:
1415  X86MaskedStoreSDNode(unsigned Opcode, unsigned Order,
1416  const DebugLoc &dl, SDVTList VTs, EVT MemVT,
1417  MachineMemOperand *MMO)
1418  : MemSDNode(Opcode, Order, dl, VTs, MemVT, MMO) {}
1419 
1420  const SDValue &getValue() const { return getOperand(1); }
1421  const SDValue &getBasePtr() const { return getOperand(2); }
1422  const SDValue &getMask() const { return getOperand(3); }
1423 
1424  static bool classof(const SDNode *N) {
1425  return N->getOpcode() == X86ISD::VMTRUNCSTORES ||
1427  }
1428  };
1429 
1430  // X86 Truncating Store with Signed saturation.
1432  public:
1433  TruncSStoreSDNode(unsigned Order, const DebugLoc &dl,
1434  SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
1435  : X86StoreSDNode(X86ISD::VTRUNCSTORES, Order, dl, VTs, MemVT, MMO) {}
1436 
1437  static bool classof(const SDNode *N) {
1438  return N->getOpcode() == X86ISD::VTRUNCSTORES;
1439  }
1440  };
1441 
1442  // X86 Truncating Store with Unsigned saturation.
1444  public:
1445  TruncUSStoreSDNode(unsigned Order, const DebugLoc &dl,
1446  SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
1447  : X86StoreSDNode(X86ISD::VTRUNCSTOREUS, Order, dl, VTs, MemVT, MMO) {}
1448 
1449  static bool classof(const SDNode *N) {
1450  return N->getOpcode() == X86ISD::VTRUNCSTOREUS;
1451  }
1452  };
1453 
1454  // X86 Truncating Masked Store with Signed saturation.
1456  public:
1457  MaskedTruncSStoreSDNode(unsigned Order,
1458  const DebugLoc &dl, SDVTList VTs, EVT MemVT,
1459  MachineMemOperand *MMO)
1460  : X86MaskedStoreSDNode(X86ISD::VMTRUNCSTORES, Order, dl, VTs, MemVT, MMO) {}
1461 
1462  static bool classof(const SDNode *N) {
1463  return N->getOpcode() == X86ISD::VMTRUNCSTORES;
1464  }
1465  };
1466 
1467  // X86 Truncating Masked Store with Unsigned saturation.
1469  public:
1471  const DebugLoc &dl, SDVTList VTs, EVT MemVT,
1472  MachineMemOperand *MMO)
1473  : X86MaskedStoreSDNode(X86ISD::VMTRUNCSTOREUS, Order, dl, VTs, MemVT, MMO) {}
1474 
1475  static bool classof(const SDNode *N) {
1476  return N->getOpcode() == X86ISD::VMTRUNCSTOREUS;
1477  }
1478  };
1479 
1480  // X86 specific Gather/Scatter nodes.
1481  // The class has the same order of operands as MaskedGatherScatterSDNode for
1482  // convenience.
1484  public:
1485  X86MaskedGatherScatterSDNode(unsigned Opc, unsigned Order,
1486  const DebugLoc &dl, SDVTList VTs, EVT MemVT,
1487  MachineMemOperand *MMO)
1488  : MemSDNode(Opc, Order, dl, VTs, MemVT, MMO) {}
1489 
1490  const SDValue &getBasePtr() const { return getOperand(3); }
1491  const SDValue &getIndex() const { return getOperand(4); }
1492  const SDValue &getMask() const { return getOperand(2); }
1493  const SDValue &getScale() const { return getOperand(5); }
1494 
1495  static bool classof(const SDNode *N) {
1496  return N->getOpcode() == X86ISD::MGATHER ||
1497  N->getOpcode() == X86ISD::MSCATTER;
1498  }
1499  };
1500 
1502  public:
1503  X86MaskedGatherSDNode(unsigned Order, const DebugLoc &dl, SDVTList VTs,
1504  EVT MemVT, MachineMemOperand *MMO)
1505  : X86MaskedGatherScatterSDNode(X86ISD::MGATHER, Order, dl, VTs, MemVT,
1506  MMO) {}
1507 
1508  const SDValue &getPassThru() const { return getOperand(1); }
1509 
1510  static bool classof(const SDNode *N) {
1511  return N->getOpcode() == X86ISD::MGATHER;
1512  }
1513  };
1514 
1516  public:
1517  X86MaskedScatterSDNode(unsigned Order, const DebugLoc &dl, SDVTList VTs,
1518  EVT MemVT, MachineMemOperand *MMO)
1519  : X86MaskedGatherScatterSDNode(X86ISD::MSCATTER, Order, dl, VTs, MemVT,
1520  MMO) {}
1521 
1522  const SDValue &getValue() const { return getOperand(1); }
1523 
1524  static bool classof(const SDNode *N) {
1525  return N->getOpcode() == X86ISD::MSCATTER;
1526  }
1527  };
1528 
1529  /// Generate unpacklo/unpackhi shuffle mask.
1530  template <typename T = int>
1532  bool Unary) {
1533  assert(Mask.empty() && "Expected an empty shuffle mask vector");
1534  int NumElts = VT.getVectorNumElements();
1535  int NumEltsInLane = 128 / VT.getScalarSizeInBits();
1536  for (int i = 0; i < NumElts; ++i) {
1537  unsigned LaneStart = (i / NumEltsInLane) * NumEltsInLane;
1538  int Pos = (i % NumEltsInLane) / 2 + LaneStart;
1539  Pos += (Unary ? 0 : NumElts * (i % 2));
1540  Pos += (Lo ? 0 : NumEltsInLane / 2);
1541  Mask.push_back(Pos);
1542  }
1543  }
1544 
1545  /// Helper function to scale a shuffle or target shuffle mask, replacing each
1546  /// mask index with the scaled sequential indices for an equivalent narrowed
1547  /// mask. This is the reverse process to canWidenShuffleElements, but can
1548  /// always succeed.
1549  template <typename T>
1551  SmallVectorImpl<T> &ScaledMask) {
1552  assert(0 < Scale && "Unexpected scaling factor");
1553  int NumElts = Mask.size();
1554  ScaledMask.assign(static_cast<size_t>(NumElts * Scale), -1);
1555 
1556  for (int i = 0; i != NumElts; ++i) {
1557  int M = Mask[i];
1558 
1559  // Repeat sentinel values in every mask element.
1560  if (M < 0) {
1561  for (int s = 0; s != Scale; ++s)
1562  ScaledMask[(Scale * i) + s] = M;
1563  continue;
1564  }
1565 
1566  // Scale mask element and increment across each mask element.
1567  for (int s = 0; s != Scale; ++s)
1568  ScaledMask[(Scale * i) + s] = (Scale * M) + s;
1569  }
1570  }
1571 } // end namespace llvm
1572 
1573 #endif // LLVM_LIB_TARGET_X86_X86ISELLOWERING_H
static SDValue LowerEXTRACT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG)
const SDValue & getIndex() const
Double shift instructions.
uint64_t CallInst * C
bool shouldTransformSignedTruncationCheck(EVT XVT, unsigned KeptBits) const override
Should we tranform the IR-optimal check for whether given truncation down into KeptBits would be trun...
static MVT getIntegerVT(unsigned BitWidth)
static SDValue LowerCallResult(SDValue Chain, SDValue InFlag, const SmallVectorImpl< CCValAssign > &RVLocs, const SDLoc &dl, SelectionDAG &DAG, SmallVectorImpl< SDValue > &InVals)
LowerCallResult - Lower the result values of a call into the appropriate copies out of appropriate ph...
TruncUSStoreSDNode(unsigned Order, const DebugLoc &dl, SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
BUILTIN_OP_END - This must be the last enum value in this list.
Definition: ISDOpcodes.h:855
A parsed version of the target data layout string in and methods for querying it. ...
Definition: DataLayout.h:111
constexpr char Align[]
Key for Kernel::Arg::Metadata::mAlign.
This represents an addressing mode of: BaseGV + BaseOffs + BaseReg + Scale*ScaleReg If BaseGV is null...
Vector comparison generating mask bits for fp and integer signed and unsigned data types...
Repeat move, corresponds to X86::REP_MOVSx.
void createUnpackShuffleMask(MVT VT, SmallVectorImpl< T > &Mask, bool Lo, bool Unary)
Generate unpacklo/unpackhi shuffle mask.
unsigned getOpcode() const
Return the SelectionDAG opcode value for this node.
LLVMContext & Context
static bool classof(const SDNode *N)
Return with a flag operand.
const SDValue & getValue() const
const SDValue & getBasePtr() const
Compute iterated dominance frontiers using a linear time algorithm.
Definition: AllocatorList.h:24
const SDValue & getScale() const
Tail call return.
Compute Double Block Packed Sum-Absolute-Differences.
A Module instance is used to store all the information related to an LLVM module. ...
Definition: Module.h:64
bool mergeStoresAfterLegalization() const override
Allow store merging after legalization in addition to before legalization.
static bool classof(const SDNode *N)
Same as call except it adds the NoTrack prefix.
static void LowerMemOpCallTo(SelectionDAG &DAG, MachineFunction &MF, SDValue Chain, SDValue Arg, SDValue PtrOff, int SPDiff, unsigned ArgOffset, bool isPPC64, bool isTailCall, bool isVector, SmallVectorImpl< SDValue > &MemOpChains, SmallVectorImpl< TailCallArgumentInfo > &TailCallArguments, const SDLoc &dl)
LowerMemOpCallTo - Store the argument to the stack or remember it in case of tail calls...
bool isMultiStoresCheaperThanBitsMerge(EVT LTy, EVT HTy) const override
Return true if it is cheaper to split the store of a merged int val from a pair of smaller values int...
X86 conditional moves.
bool isScalarInteger() const
Return true if this is an integer, but not a vector.
Definition: ValueTypes.h:146
This class represents a function call, abstracting a target machine&#39;s calling convention.
unsigned getVectorNumElements() const
Function Alias Analysis Results
This instruction constructs a fixed permutation of two input vectors.
bool hasFnAttribute(Attribute::AttrKind Kind) const
Return true if the function has the attribute.
Definition: Function.h:321
Fast - This calling convention attempts to make calls as fast as possible (e.g.
Definition: CallingConv.h:43
unsigned const TargetRegisterInfo * TRI
A debug info location.
Definition: DebugLoc.h:34
bool isInteger() const
Return true if this is an integer or a vector integer type.
Definition: ValueTypes.h:141
static bool classof(const SDNode *N)
SSE4A Extraction and Insertion.
static bool classof(const SDNode *N)
An instruction for reading from memory.
Definition: Instructions.h:168
an instruction that atomically reads a memory location, combines it with another value, and then stores the result back.
Definition: Instructions.h:681
Bitwise logical ANDNOT of floating point values.
AtomicExpansionKind
Enum that specifies what an atomic load/AtomicRMWInst is expanded to, if at all.
bool isCalleePop(CallingConv::ID CallingConv, bool is64Bit, bool IsVarArg, bool GuaranteeTCO)
Determines whether the callee is required to pop its own arguments.
This operation implements the lowering for readcyclecounter.
NodeType
ISD::NodeType enum - This enum defines the target-independent operators for a SelectionDAG.
Definition: ISDOpcodes.h:39
X86MaskedGatherScatterSDNode(unsigned Opc, unsigned Order, const DebugLoc &dl, SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
A convenience struct that encapsulates a DAG, and two SDValues for returning information from TargetL...
X86 compare and logical compare instructions.
MaskedTruncUSStoreSDNode(unsigned Order, const DebugLoc &dl, SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
bool isFloatingPoint() const
Return true if this is a FP or a vector FP type.
Definition: ValueTypes.h:136
static GCMetadataPrinterRegistry::Add< OcamlGCMetadataPrinter > Y("ocaml", "ocaml 3.10-compatible collector")
Extract an 8-bit value from a vector and zero extend it to i32, corresponds to X86::PEXTRB.
A description of a memory reference used in the backend.
X86StoreSDNode(unsigned Opcode, unsigned Order, const DebugLoc &dl, SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
Bitwise Logical AND NOT of Packed FP values.
This class consists of common code factored out of the SmallVector class to reduce code duplication b...
Definition: APFloat.h:42
Base class for the full range of assembler expressions which are needed for parsing.
Definition: MCExpr.h:36
This provides a uniform API for creating instructions and inserting them into a basic block: either a...
Definition: IRBuilder.h:743
This instruction implements SINT_TO_FP with the integer source in memory and FP reg result...
static SDValue LowerRETURNADDR(SDValue Op, SelectionDAG &DAG, const SparcTargetLowering &TLI, const SparcSubtarget *Subtarget)
The MachineFrameInfo class represents an abstract stack frame until prolog/epilog code is inserted...
This class defines information used to lower LLVM code to legal SelectionDAG operators that the targe...
Integer horizontal add/sub.
This represents a list of ValueType&#39;s that has been intern&#39;d by a SelectionDAG.
Copies a 64-bit value from the low word of an XMM vector to an MMX vector.
void assign(size_type NumElts, const T &Elt)
Definition: SmallVector.h:423
Context object for machine code objects.
Definition: MCContext.h:63
static SDValue LowerINSERT_VECTOR_ELT(SDValue Op, SelectionDAG &DAG)
Copies a 32-bit value from the low word of a MMX vector to a GPR.
This is a fast-path instruction selection class that generates poor code and doesn&#39;t support illegal ...
Definition: FastISel.h:67
X86 FP SETCC, similar to above, but with output as an i1 mask and with optional rounding mode...
Return from interrupt. Operand 0 is the number of bytes to pop.
This contains information for each constraint that we are lowering.
ArrayRef - Represent a constant reference to an array (0 or more elements consecutively in memory)...
Definition: APInt.h:33
An instruction for storing to memory.
Definition: Instructions.h:310
static bool classof(const SDNode *N)
X86 FP SETCC, implemented with CMP{cc}SS/CMP{cc}SD.
const SDValue & getBasePtr() const
virtual unsigned getInlineAsmMemConstraint(StringRef ConstraintCode) const
Floating point horizontal add/sub.
unsigned getInlineAsmMemConstraint(StringRef ConstraintCode) const override
amdgpu Simplify well known AMD library false Value * Callee
Bitwise logical XOR of floating point values.
uint16_t MCPhysReg
An unsigned integer type large enough to represent all physical registers, but not necessarily virtua...
static bool classof(const SDNode *N)
const SDValue & getMask() const
Machine Value Type.
static SDValue LowerVASTART(SDValue Op, SelectionDAG &DAG)
The instances of the Type class are immutable: once they are created, they are never changed...
Definition: Type.h:46
This instruction implements an extending load to FP stack slots.
This is an important class for using LLVM in a threaded context.
Definition: LLVMContext.h:69
Insert any element of a 4 x float vector into any element of a destination 4 x floatvector.
unsigned getScalarSizeInBits() const
size_t size() const
size - Get the array size.
Definition: ArrayRef.h:149
This is an important base class in LLVM.
Definition: Constant.h:42
Repeat fill, corresponds to X86::REP_STOSx.
static bool is64Bit(const char *name)
LoadExtType
LoadExtType enum - This enum defines the three variants of LOADEXT (load with extension).
Definition: ISDOpcodes.h:912
bool storeOfVectorConstantIsCheap(EVT MemVT, unsigned NumElem, unsigned AddrSpace) const override
Return true if it is expected to be cheaper to do a store of a non-zero vector constant with the give...
X86 conditional branches.
Insert the lower 16-bits of a 32-bit value to a vector, corresponds to X86::PINSRW.
Commutative FMIN and FMAX.
static SDValue LowerFRAMEADDR(SDValue Op, SelectionDAG &DAG, const SparcSubtarget *Subtarget)
On Darwin, this node represents the result of the popl at function entry, used for PIC code...
bool convertSetCCLogicToBitwiseLogic(EVT VT) const override
Use bitwise logic to make pairs of compares more efficient.
static SDValue LowerUINT_TO_FP(SDValue Op, SelectionDAG &DAG, const SparcTargetLowering &TLI, bool hasHardQuad)
const SDValue & getValue() const
static SDValue LowerDYNAMIC_STACKALLOC(SDValue Op, SelectionDAG &DAG, const SparcSubtarget *Subtarget)
lazy value info
std::vector< ArgListEntry > ArgListTy
Extended Value Type.
Definition: ValueTypes.h:34
TargetRegisterInfo base class - We assume that the target defines a static array of TargetRegisterDes...
This structure contains all information that is necessary for lowering calls.
These operations represent an abstract X86 call instruction, which includes a bunch of information...
Floating point max and min.
static SDValue LowerSETCCCARRY(SDValue Op, SelectionDAG &DAG)
TruncSStoreSDNode(unsigned Order, const DebugLoc &dl, SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
bool isScalarFPTypeInSSEReg(EVT VT) const
Return true if the specified scalar FP type is computed in an SSE register, not on the X87 floating p...
Copies a GPR into the low 32-bit word of a MMX vector and zero out the high word. ...
CallingConv::ID getCallingConv() const
getCallingConv()/setCallingConv(CC) - These method get and set the calling convention of this functio...
Definition: Function.h:213
This is used to represent a portion of an LLVM function in a low-level Data Dependence DAG representa...
Definition: SelectionDAG.h:222
unsigned getMaxSupportedInterleaveFactor() const override
Get the maximum supported factor for interleaved memory accesses.
Provides information about what library functions are available for the current target.
X86 Read Time-Stamp Counter and Processor ID.
CCValAssign - Represent assignment of one arg/retval to a location.
AddressSpace
Definition: NVPTXBaseInfo.h:22
unsigned getMemcmpEqZeroLoadsPerBlock() const override
Allow multiple load pairs per block for smaller and faster code.
X86MaskedScatterSDNode(unsigned Order, const DebugLoc &dl, SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
This is an abstract virtual class for memory operations.
Wrapper class for IR location info (IR ordering and DebugLoc) to be passed into SDNode creation funct...
Bit scan reverse.
Floating point reciprocal-sqrt and reciprocal approximation.
static const int FIRST_TARGET_MEMORY_OPCODE
FIRST_TARGET_MEMORY_OPCODE - Target-specific pre-isel operations which do not reference a specific me...
Definition: ISDOpcodes.h:862
const SDValue & getValue() const
Represents one node in the SelectionDAG.
X86 bit-test instructions.
static bool Enabled
Definition: Statistic.cpp:51
const Function & getFunction() const
Return the LLVM function that this machine code represents.
static bool classof(const SDNode *N)
MaskedTruncSStoreSDNode(unsigned Order, const DebugLoc &dl, SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
Class for arbitrary precision integers.
Definition: APInt.h:70
static SDValue LowerVAARG(SDValue Op, SelectionDAG &DAG)
static bool classof(const SDNode *N)
This instruction implements FP_TO_SINT with the integer destination in memory and a FP reg source...
Bit scan forward.
const char * getClearCacheBuiltinName() const override
Intel processors have a unified instruction and data cache.
amdgpu Simplify well known AMD library false Value Value * Arg
bool ShouldShrinkFPConstant(EVT VT) const override
If true, then instruction selection should seek to shrink the FP constant of the specified type to a ...
Representation of each machine instruction.
Definition: MachineInstr.h:64
static unsigned getScalingFactorCost(const TargetTransformInfo &TTI, const LSRUse &LU, const Formula &F, const Loop &L)
bool isVector() const
Return true if this is a vector value type.
Definition: ValueTypes.h:151
Insert the lower 8-bits of a 32-bit value to a vector, corresponds to X86::PINSRB.
LLVM_NODISCARD bool empty() const
Definition: SmallVector.h:56
A wrapper node for TargetConstantPool, TargetJumpTable, TargetExternalSymbol, TargetGlobalAddress, TargetGlobalTLSAddress, MCSymbol and TargetBlockAddress.
Bitwise logical AND of floating point values.
#define I(x, y, z)
Definition: MD5.cpp:58
#define N
FunctionLoweringInfo - This contains information that is global to a function that is used when lower...
static bool classof(const SDNode *N)
X86MaskedStoreSDNode(unsigned Opcode, unsigned Order, const DebugLoc &dl, SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
uint32_t Size
Definition: Profile.cpp:47
LOCK-prefixed arithmetic read-modify-write instructions.
Extract a 16-bit value from a vector and zero extend it to i32, corresponds to X86::PEXTRW.
bool hasVectorBlend() const override
Return true if the target has a vector blend instruction.
Blend where the selector is an immediate.
X86MaskedGatherSDNode(unsigned Order, const DebugLoc &dl, SDVTList VTs, EVT MemVT, MachineMemOperand *MMO)
This instruction implements a truncating store to FP stack slots.
Combined add and sub on an FP vector.
assert(ImpDefSCC.getReg()==AMDGPU::SCC &&ImpDefSCC.isDef())
LegalizeTypeAction
This enum indicates whether a types are legal for a target, and if not, what action should be used to...
This instruction grabs the address of the next argument from a va_list.
LLVM Value Representation.
Definition: Value.h:73
Bitwise logical OR of floating point values.
Dynamic (non-constant condition) vector blend where only the sign bits of the condition elements are ...
X86 Read Performance Monitoring Counters.
const SDValue & getBasePtr() const
std::underlying_type< E >::type Mask()
Get a bitmask with 1s in all places up to the high-order bit of E&#39;s largest value.
Definition: BitmaskEnum.h:81
static SDValue LowerSINT_TO_FP(SDValue Op, SelectionDAG &DAG, const SparcTargetLowering &TLI, bool hasHardQuad)
IRTranslator LLVM IR MI
bool isZeroNode(SDValue Elt)
Returns true if Elt is a constant zero or floating point constant +0.0.
StringRef - Represent a constant reference to a string, i.e.
Definition: StringRef.h:49
bool isOffsetSuitableForCodeModel(int64_t Offset, CodeModel::Model M, bool hasSymbolicDisplacement=true)
Returns true of the given offset can be fit into displacement field of the instruction.
const SDValue & getPassThru() const
bool hasBitPreservingFPLogic(EVT VT) const override
Return true if it is safe to transform an integer-domain bitwise operation into the equivalent floati...
Compute Sum of Absolute Differences.
Scalar intrinsic floating point max and min.
MVT getScalarShiftAmountTy(const DataLayout &, EVT VT) const override
EVT is not used in-tree, but is used by out-of-tree target.
Unlike LLVM values, Selection DAG nodes may return multiple values as the result of a computation...
void scaleShuffleMask(int Scale, ArrayRef< T > Mask, SmallVectorImpl< T > &ScaledMask)
Helper function to scale a shuffle or target shuffle mask, replacing each mask index with the scaled ...
constexpr char Args[]
Key for Kernel::Metadata::mArgs.
FastISel * createFastISel(FunctionLoweringInfo &funcInfo, const TargetLibraryInfo *libInfo)
BRIND node with NoTrack prefix.
Shuffle 16 8-bit values within a vector.
This file describes how to lower LLVM code to machine code.
Special wrapper used under X86-64 PIC mode for RIP relative displacements.